{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression With Non-Linear Boundary Demo\n",
    "\n",
    "_Source: 🤖[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) repository_\n",
    "\n",
    "> ☝Before moving on with this demo you might want to take a look at:\n",
    "> - 📗[Math behind the Logistic Regression](https://github.com/trekhleb/homemade-machine-learning/tree/master/homemade/logistic_regression)\n",
    "> - ⚙️[Logistic Regression Source Code](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py)\n",
    "\n",
    "**Logistic regression** is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.\n",
    "\n",
    "Logistic Regression is used when the dependent variable (target) is categorical.\n",
    "\n",
    "For example:\n",
    "\n",
    "- To predict whether an email is spam (`1`) or (`0`).\n",
    "- Whether online transaction is fraudulent (`1`) or not (`0`).\n",
    "- Whether the tumor is malignant (`1`) or not (`0`).\n",
    "\n",
    "> **Demo Project:** In this example we will try to classify microchips into to categories (`valid` and `invalid`) based on two artifical parameters `param_1` and `param_2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To make debugging of logistic_regression module easier we enable imported modules autoreloading feature.\n",
    "# By doing this you may change the code of logistic_regression library and all these changes will be available here.\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "# Add project root folder to module loading paths.\n",
    "import sys\n",
    "sys.path.append('../..')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Dependencies\n",
    "\n",
    "- [pandas](https://pandas.pydata.org/) - library that we will use for loading and displaying the data in a table\n",
    "- [numpy](http://www.numpy.org/) - library that we will use for linear algebra operations\n",
    "- [matplotlib](https://matplotlib.org/) - library that we will use for plotting the data\n",
    "- [logistic_regression](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py) - custom implementation of logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import 3rd party dependencies.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Import custom logistic regression implementation.\n",
    "from homemade.logistic_regression import LogisticRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the Data\n",
    "\n",
    "In this demo we will use artificial dataset in which `param_1` and `param_2` produce non-linear decision boundary (see the plot below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>param_1</th>\n",
       "      <th>param_2</th>\n",
       "      <th>validity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.699560</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.684940</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.692250</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.502190</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.465640</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>-0.524770</td>\n",
       "      <td>0.209800</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>-0.398040</td>\n",
       "      <td>0.034357</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>-0.305880</td>\n",
       "      <td>-0.192250</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.016705</td>\n",
       "      <td>-0.404240</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.131910</td>\n",
       "      <td>-0.513890</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    param_1   param_2  validity\n",
       "0  0.051267  0.699560         1\n",
       "1 -0.092742  0.684940         1\n",
       "2 -0.213710  0.692250         1\n",
       "3 -0.375000  0.502190         1\n",
       "4 -0.513250  0.465640         1\n",
       "5 -0.524770  0.209800         1\n",
       "6 -0.398040  0.034357         1\n",
       "7 -0.305880 -0.192250         1\n",
       "8  0.016705 -0.404240         1\n",
       "9  0.131910 -0.513890         1"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the data.\n",
    "data = pd.read_csv('../../data/microchips-tests.csv')\n",
    "\n",
    "# Print the data table.\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Data\n",
    "\n",
    "Let's plot the data on 2D-plane and split it by two clasess (`valid` and `invalid`) to see the distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEXCAYAAABlI9noAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xu8XGV97/HPl2vCQUkIiGTvhISSIkhSoluUxnoBNIgKOZbGxFqDQqMtXl56ioRDD1KsL6K2ckrxUg5S8MZFqhCrgEBAK4qwU8LVxgQisjcgmBiqEkKA3/ljrSErk5m95/KstZ4183u/Xvs1M2utmXlm7b3nt57b75GZ4ZxzzoWwU9kFcM451zs8qDjnnAvGg4pzzrlgPKg455wLxoOKc865YDyoOOecC8aDiusrkr4k6f8U/J4m6aAm+/5c0veLLI9zeZLPU3G9QNIvgKnAVDP7dWb7ncDhwEwz+0VJZTNglpmtK+C9/gS4tvYQ2AP4feaQQ83slx287gRgMzDNzEa6LqjrWV5Tcb1kPbC49kDSbJIv1a4oUYn/FTP7DzPb08z2BF6ebp5U29ZJQHGuHZX4R3GuRV8F3pN5vAT4SvYASZdI+vvM4xMkrZb035IekHRsuv0WSZ+SdCvwFHCgpKmSVkjaKGmdpL/MvM7Okv53+hq/lbRK0rTMWx8jaa2kTZI+L0np806S9KPM65ikD0t6UNKvJX22FtAkHSTpB5KeTPdd0clJkrS3pK9IekzSw5I+kXmPl0n6UfoeT0iqnb8fprdrJP1O0gJJL5V0XfqZNkha2Ul5XG/ZpewCOBfQbcBfSDoE+DmwCJgH/H2jgyUdQRJ0TgRuAvYHXpQ55C+AtwBrSJqSbgLuJWlmexlwg6QHzGwl8DGSWtJx6XvPIQlGNW8DXgW8GFgFfAe4rsnn+J/AELAncGP6/hcBnwS+D7wR2C09phNfB9YBBwJ7Ad8DfgFcCpwLXA38CTABeEX6nNeRNH8dXGv+knReWra3kVygvqbD8rge4jUV12tqtZU3AT8DRsc49mTgYjO7wcyeN7NRM/uvzP5LzOw+M3sWeClJgDrdzJ42s9UkX/S1mtEpwN+a2RpL3GVmGzKvtdzMNqXNTzeT9PM082kz25ge+3/Z1qS3FTiApN/oaTP7UdNXaELSASQB4mNm9pSZPQqcTxKAa+8xA3ipmW02s1vHeLmtJAF2upk9Y2Y/HONY1yc8qLhe81XgXcBJ1DV9NTANeGCM/Q9n7k8FNprZbzPbHgIGWnytxzL3nyKphbTyvg+l7w3wcZIa0+2S7pP0vjFeo5kDSGogT6TNVpuAfwL2S/d/lKQf6k5Jd0t69xiv9SngEeDmtDnwYx2Ux/UYb/5yPcXMHpK0nqQZ6uRxDn8Y+IOxXi5z/xFgb0kvygSW6WyrCdVe6972S72DacB9mfd4BMDMHgP+EkDSa4EbJf2wzVFlDwO/AyZbg6GfZjYKvC/t83k98H1JPwR+1eDYJ4GPAB+R9EckweWn49RuXI/zmorrRScDR5nZ78c57svAeyUdLWknSQOSXtboQDN7GPgxcK6kCZLmpO/ztfSQi4BPSpqVjhabI2lKh+U/TdLktKP/I8AVAJL+TNJgesxvSILe8+28sJmtJ+l7+oykF6Wfe1YapJD0TklT04CzKX3ac2a2BXiSpB+G9NjjJR2YBqAngefaLY/rPR5UXM8xswfMbLiF424H3gucR/Kl+AOS5qFmFpP0NzwCfBv4hJndmO77HHAlSUf6f5MErIkdfoRrSDrzVwPfTV8Lko7+n0r6HbAC+IiZPdjB6y8GJgH/BWwkCVq15q8jgVXpe3wTWJrWXgDOAr6ZNpsdDxxC0j/0W5LRYf9gZj/poDyuh/jkR+ciUuRESefy4DUV55xzwXhQcc45F4w3fznnnAvGayrOOeeC6bt5Kvvss4/NmDGj7GI451ylrFq16tdmtu94x/VdUJkxYwbDw+OONnXOOZch6aFWjvPmL+ecc8F4UHHOOReMBxXnnHPB9F2finPOlWHr1q2MjIzw9NNPl12UMU2YMIHBwUF23XXXjp7vQcU55wowMjLCi170ImbMmEG68Gd0zIwNGzYwMjLCzJkzO3oNb/5yzrkCPP3000yZMiXagAIgiSlTpnRVm/Kg4pxzBYk5oNR0W0Zv/nJ97+o7R/ns9Wt4ZNNmpk6ayGnzD2bB3IHxn+ic24HXVFxfu/rOUc741j2MbtqMAaObNnPGt+7h6jvHWtreueq67rrrOPjggznooINYvnx58Nf3oOL62mevX8Pmrc9tt23z1uf47PVrSipRvq6+c5R5y1cyc9l3mbd8pQfPPvPcc89x6qmncu2113L//fdz2WWXcf/99wd9D2/+cn3tkU2b29peZbVaWS2I1mplgDf3RSiPZtnbb7+dgw46iAMPTFaFXrRoEddccw2HHnpoiCIDXlNxfW7qpMYr/jbbXmX9ViursryaZUdHR5k2bdoLjwcHBxkdDVtb9aDi+tpp8w9m4q47b7dt4q47c9r8g0sqUX76qVZWdVW+APCg4vragrkDnPuO2QxMmoiAgUkTOfcds3uyOaifamVVl9cFwMDAAA8//PALj0dGRhgYCPu37n0qru8tmDvQk0Gk3mnzD96uTwV6t1ZWdVMnTWS0QQDp9gLgVa96FWvXrmX9+vUMDAxw+eWX841vfKOr16znNRXn+kQ/1cqqLq9m2V122YULLriA+fPnc8ghh7Bw4UJe/vKXd/WaO7xH0FdzzkWtX2plVVf7HeUxKfe4447juOOO6/p1mvGg4pxzEarqBUDpzV+SLpb0uKR7m+yXpPMlrZN0t6RXZPYtkbQ2/VlSXKl7i0+Ia4+fL+eai6GmcglwAfCVJvvfAsxKf14NfBF4taS9gU8AQ4ABqyStMLPf5F7iHtKPE+K6mVTWj+fLuXaUXlMxsx8CG8c45ATgK5a4DZgkaX9gPnCDmW1MA8kNwLH5l7i3VHk8fCe6nVTWb+fLuXaVHlRaMAA8nHk8km5rtn0HkpZKGpY0/MQTT+RW0Crqtwlx3QaFfjtfzrWrCkGla2Z2oZkNmdnQvvvuW3ZxohLdhLi7r4TzDoOzJyW3d18Z9OW7DQrRnS/nIlOFoDIKTMs8Hky3Ndvu2hBVmpK7r4TvfBiefBiw5PY7Hw4aWLoNClGdL+fa9L73vY+XvOQlHHbYYbm9RxWCygrgPekosNcAT5rZo8D1wJslTZY0GXhzus21IaoJcTedA1vragxbNyfbA+k2KER1vpxr00knncR1112X63uUPvpL0mXAG4B9JI2QjOjaFcDMvgR8DzgOWAc8Bbw33bdR0ieBO9KXOsfMxurwd01EMx7+yZH2tncgxKSyss6Xr1DZZ+6+MrmgenIE9hqEo8+COQu7esnXve51/OIXvwhTviZKDypmtnic/Qac2mTfxcDFeZTLtSfIF95eg2nTV4PtARUVFEIGAR/K3GdqTcG1mnutKRi6Dix5q0Lzl4tcsLUfjj4Ldq3r29h1YrK9YkKvh+FDmftMAU3BefGg0qdCzgoP9oU3ZyG8/XzYaxqg5Pbt50d/ZdZI6CDgQ5n7TAFNwXkpvfnLFS90U0rQL7w5CysZROqFDgJ5pUJ3kSqoKTgPXlPpQ6Gvon3uxo5CnxMfyjy+nsrJllNT8OLFiznyyCNZs2YNg4ODfPnLX+7q9RrxmkofCn0V7Ys/7Sj0OckzFXov6LmBDLXaeuDRX5dddlmAwo3Ng0ofCt2U4l94O8rjnEQz9DtCY9W+K3vOKtoU7EGlD+VRs/AvvB35OSmOD2SIh/ep9CGfFe56TVX69ZJpd3HrtoxeU+lTfhXtekkV+vUmTJjAhg0bmDJlCpLKLk5DZsaGDRuYMGFCx6/hQcU515YY08VUoV9vcHCQkZERYl9+Y8KECQwOdj50WVWojoU0NDRkw8PDZRfDVUEOuZeqrn6UFSQ1Am8+7X2SVpnZ0HjHeZ+Kc40UkIa/ijxdjBuPBxXXW0It8lXh3Et58lFWbjweVFzvCFm7qHDupTxVZZSVK48HFVee0EsHh6xdNMuxFCj3UuiUIkWlKPF0MW48HlRcOfLoswhZu8gxDX/otPihX28sPsfJjceHFLtyjFWr6HSEVcjMrjnlXoLwKUWKTlESYo5TjMOSXRilBxVJxwL/BOwMXGRmy+v2nwe8MX24B/ASM5uU7nsOuCfd90szO76YUruu5dFncfRZ26+WB93VLnLKvRS6s7tqnec9l/zRbafU5i9JOwOfB94CHAoslnRo9hgz+6iZHW5mhwP/DHwrs3tzbZ8HlIrJo8+iIot8he7srlrnuQ9L7m1l96kcAawzswfN7BngcuCEMY5fDOSfu9nlL68+izkL4aP3wtmbktvIAgqE7+yuWud5yJpVT62h0iPKDioDQLYRfCTdtgNJBwAzgZWZzRMkDUu6TdKCZm8iaWl63HDsKRL6RkVqFXkI3dldtc7zUDWrIgcouNaVmqZF0onAsWZ2Svr4L4BXm9kHGxx7OjBoZh/KbBsws1FJB5IEm6PN7IGx3rOf0rTk2RnqHa2uU6FSvcxbvrLhukADkyZy67KjgpTVbdNqmpayO+pHgWmZx4PptkYWAadmN5jZaHr7oKRbgLnAmEGlX+TZGRptR2tMubpiKktkQiV/rNoAhX5RdlC5A5glaSZJMFkEvKv+IEkvAyYDP8lsmww8ZWZbJO0DzAM+U0ipKyDPYaalrrLX7Mu6Nu+lNvKrNu8Fiv8yj6ksrSghAIYYlhx6BVMXRql9Kmb2LPBB4HrgZ8CVZnafpHMkZUdzLQIut+3b6g4BhiXdBdwMLDez+4sqewh5djLmeRVX2hXiWBMmO5lNH3pGf02V8oZVOHFm1QYo9IuyayqY2feA79VtO6vu8dkNnvdjYHauhctR3k1IeV7FlXaFONaXdbvzXvKsTVQpb1joSagF1nqqsIZKPyo9qPSrvJuQ8lwJr7RV9sb6sm53Nn0eM/qz7xlqZn/eQgbAEpr9YljB1AetbK/sIcV9K+8mpDyHmZY2hHWsCZPtznvJszaRY96w4EJOQq1Ss18gPqx5R15TKUkRTUh5XsWVcoU4VhqWOQvhl7fBqkvAngPtDH/0ruZXyHnWJnLMGxZcyNQ2kTb75VmTKHXQSqS8plKS6DoZ8+q0DmmsCZN3Xwl3fSMJKJDc3vWN5p8j79pEBWb2A2Enoea8XEAn8q5J+LDmHXlNpSRRdTJWaQhssySP7faRVKk2kbdQiTNDJ/QMIO+ahA9r3pEHlRLF0MkI5NtpXZROml5yykIcu9yagyIM1HnXJEobtBIxDyou2rbwtlRpxFWJcs+GEFmgzrsmEVWLQyQ8qFRVyPkAvfCFHGHTS4x6pWO51dpWETWJaFocIuEd9VUUehZ0lYbANtPHWY/b0ajZ5/idfsQVT/1l3IM0MtrpfK9aBudeUGqW4jL0RJbi8w5rUrOYlow06oQnQIxDzr+H+sy+x+/0I5bvehF76JltB+06MeqA7NmJy1GVLMWuE3n0gUTWFt6XChiFV98c9PFdrtw+oED0gzR8GG/cvPmriiKcD1BJsc3NKWBGen1z0NSdNjQ+MOJBGlVbPrnfeFCpol7oAylKs8ARY3begkbhLZg7wK3LjmL98reyUwUvUKKbOOy240GlYEHS3XundGtCp8rPW7Mvcu2UX22qghco3vkeN++oL1CoZVRdi8Ya0PDkCNDob19JapUy1PepNNKsE72bDn4fpOFa4B31EeqVOQKVETJVfhHqZ6Rrp225zGoadaJ328HvgzRcQN78VSAftVKwkKnyi5JNRGnPNz6mPljG2JTn+lbpQUXSsZLWSFonaVmD/SdJekLS6vTnlMy+JZLWpj9Lii15+/IYtZLnksRti2001ViBowr9Uq12ovdCmp0IRfW/VSGlNn9J2hn4PPAmYAS4Q9KKBmvNX2FmH6x77t7AJ4AhksbxVelzf1NA0TsSOmVE7nmc2hFjpuPxEhzG3uzTauqZGJvyuhFBH09U/1sVU3ZN5QhgnZk9aGbPAJcDJ7T43PnADWa2MQ0kNwDH5lTOIEKPWhmrj6ZwsTbBVGVdk0ZarU3F2pTXiUiGekf1v1UxZXfUDwDZS6wR4NUNjvtTSa8Dfg581MwebvLcht/OkpYCSwGmT58eoNidC5l8ruM+mjyuBLtpgongyjRardSmIkw537FIlmHw/s/OlR1UWvEd4DIz2yLp/cClQFsJfszsQuBCSIYUhy9iOTpK651XM1WnTTAxNptVUexNea2KpH/IF9/qXNnNX6PAtMzjwXTbC8xsg5ltSR9eBLyy1ef2uo5mFufVTNVpE0yszWauHJHM8PdZ+50rO6jcAcySNFPSbsAiYEX2AEn7Zx4eD/wsvX898GZJkyVNBt6cbusbHfXR5HUl2OloqkiuTF0kyugfajBq0Wftd67U5i8ze1bSB0mCwc7AxWZ2n6RzgGEzWwF8WNLxwLPARuCk9LkbJX2SJDABnGNmGwv/ECVru48mz5FCnTTB9NrIJdedovuHxmh+XTB3oQeRDnialn7TKBVImetnxFYe11/yWJuoR7WapqXs5i9XtNgm/cVWHtdfvPk1uCqM/nKhxTZSKLbyjMeHQPcOb34NzmsqzrUjksl5LpBemjgaCQ8qzrXDh0D3Fm9+Dc6bv5xrh7fB956qNb9GzoOKc+3wNngX2NV3jvLZ69fwyKbNTJ00kdPmH1zpocze/OWqp8wU+94G7wKqZUMe3bQZY1s25Cqn2feg4qql7I5yb4N3AfViNmRv/qoKH8aaiCGLrbfBu0B6MRuy11SqoOyr85h4R7nrIXmsBlu2cYOKpJ0lvV/SJyXNq9v3t/kVLS6lLi1apWGsefd3RJLF1rkQejEbcis1lX8BXg9sAM6X9LnMvnfkUqrIlN6ZVpWr8yJqVN5R7npIL2ZDbqVP5QgzmwMg6QLgC5K+BSwGlGfhYjFWZ1ohv/yqDGMtor+jl1Y5dI6wq8HGoJWgslvtjpk9CyyVdBawEtgzr4LFpJPOtKBjz48+q3Em39iuzouqUXlHuXPRaiWoDEs61syuq20ws3MkPQJ8Mb+ixaPdpUVrzWW12k2tuQzoLLBU5eq8KjUq5wLqtcmL3Rq3T8XM3p0NKJntF5nZrrXHkt4UunCxaLczLZex53MWJus7nL0puY0toID3d7h8lDnZdRyl97dGKOSQ4k8HfK2otNuZ1otjz1viEwNdaJEPp+/FyYvdCjn5saNOe0nHAv9EspzwRWa2vG7/x4BTSJYTfgJ4n5k9lO57DrgnPfSXZnZ8h2UfVzudae02l/UU7+9wIcUw2XUMfXsBOYaQNZW21yWWtDPweeAtwKHAYkmH1h12JzCUjkC7CvhMZt9mMzs8/cktoLSrF8eeO1eKyIfT9+LkxW6VPaP+CGCdmT1oZs8AlwMnZA8ws5vN7Kn04W1A9L2+vTj23LlSRD7Z1S8gdxSy+esXHTxnAMgOFxoBXj3G8ScD12YeT5A0TNI0ttzMrm70JElLgaUA06dP76CY7eupseeed8yVJfLh9LX/cR/9tU3LQSVtqnorMCP7PDP7XHqb6+x6Se8Ghkhm99ccYGajkg4EVkq6x8weqH+umV0IXAgwNDTUdjNdz+gkONQ6Smv/1LWOUvDA4vJXgeH0PXUBGUA7NZXvAE+TdIw/H+j9R4FpmceD6bbtSDoGOBN4vZltqW03s9H09kFJtwBzgR2CiqPz4BB5R6nrAz74o1LaCSqDtXQtAd0BzJI0kySYLALelT1A0lyS/GPHmtnjme2TgafMbIukfYB5bN+J77I6DQ6Rd5Q65+LSTkf9tZLeHPLN07QvHwSuB34GXGlm90k6R1JtNNdnSdLBfFPSakkr0u2HkMz2vwu4maRP5f6Q5espnQaHyDtKnXNxaaemchvwbUk7AVtJ5qWYmb24mwKY2feA79VtOytz/5gmz/sxMLub9+4rnaZQibGj1AcOOBetdmoqnwOOBPYwsxeb2Yu6DSiuQJ2mUIltlnzkM6xdSSJO5dJv2qmpPAzca2b9O3qqyroZRRNTR6kPHHD1fIRiVNoJKg8Ct0i6FsiOwPpc86e4qMQUHDrlAwdcPb/QiEo7QWV9+rMbmTVWnCuUp9d39fxCIyotBxUz+7s8C+JcS2IcOODK5RcaUWm5o17SvpI+K+l7klbWfvIsnHM7iG3ggCufr+MTlXaav74OXAG8DfgAsIQkFb1zxeqFviEXTgVSuZSljFUp2wkqU8zsy5I+YmY/AH4g6Y68Cuaccy3zC40dBF/WvEXtBJWt6e2jkt4KPALsHb5Izjnnuq1ljLUqZSxB5e8l7QX8L+CfgRcDH82lVM4518dC1DLKWpWypY76NO39LDN70szuNbM3mtkrzWzFuE92zjnXlrFqGa0qa1XKloKKmT0HLM61JM4554AwtYyyVqVsp/nrVkkXkIwA+31to5n9Z/BS9YkyRmY45+I3ddJERhsEkHZqGWWtStlOUDk8vT0ns82Ao8IVp3+UNTIjKM8W7FwuTpt/8HbfD9BZLaOMVSnbmVH/xjwL0m/KGpkRTLdJ/DwgOddUWbWMENqpqZAOJX45MKG2zczOaf4M10xZIzOC6SaJn2eV7Q9+4dCWRs3hty6rXkNQO2lavgS8E/gQyQJdfwYckFO5el6wkRllrSPRTRK/sQKS6w2+7k1bas3ho5s2Y2xrDr/6ztGyi9a2dhbp+mMzew/wmzS55JHAH3ZbAEnHSlojaZ2kZQ327y7pinT/TyXNyOw7I92+RtL8bstSpCAjM8r8x+1mmWHPKtv7/MKhLSGGEMeinaBS+wt5StJUkhn2+3fz5un8l88DbwEOBRZLOrTusJNJAtlBwHnAp9PnHgosImmOOxb4Qvp6lbBg7gDnvmM2A5MmImBg0kTOfcfs9tpMy/zH7SaJn6973/v8wqEtlW8Oz2inT+XfJU0CPgOsSrdd1OX7HwGsM7MHASRdDpwA3J855gTg7PT+VcAFkpRuv9zMtgDrJa1LX+8nXZapMF2PzCjzH7ebJH6dpq/3Nvrq8HT0bQkxhDgW7QSVfwD+CvgTki/u/wC+2OX7D5AsU1wzAry62TFm9qykJ4Ep6fbb6p7b8Bta0lJgKcD06dO7LHJEyv7H7TSJXycByTv3q6XgdW+qPucr1BDiGLQTVC4Ffgucnz5+F/AVIPr/aDO7ELgQYGhoyEouTjhVXrCq3YDkS8ZWS4Hp6HthzleVhxDXayeoHGZm2f6OmyXd3/To1owC0zKPB9NtjY4ZkbQLsBewocXn9rZ+WkfC2+irp6B09JWf85UqY6JiHtoJKv8p6TVmdhuApFcDw12+/x3ALEkzSQLCIpIaUNYKkgXBfgKcCKw0M5O0AviGpM8BU4FZwO1dlqd6Kr6ORMvNFmU39blo9VIndy9oJ6i8EvixpF+mj6cDayTdA5iZzWn3zdM+kg8C1wM7Axeb2X2SzgGG0yzIXwa+mnbEbyQJPKTHXUnSqf8scGqa+LLnVb39uKatZosqN/W5XPVSJ3cvkFlrXQySxpzoaGYPBSlRzoaGhmx4uNsKVmdCBIP6L2JIOvTaHo4cgXnLVzb8MhiYNLHxTGIf/eUaiPV/olcu/mokrTKzofGOayf3VyWCRqxCdSb2SvsxdNBsUfGmPpePGDu5e2HwQKfayv3lOhcqGPRS+7E3W7hQYuvk7qWLv3a1M6PedSFUMChrNbc8lLWIkHN566WLv3Z5UClIqGDQS1/EQVLVOBehXrr4a5c3fxUk5KI7EFf7cTdia7ZwLoRemiHfLg8qBQkZDPL8Iu61ESuuz0QyQrDXLv7a0fKQ4l5R5pDi2MU6NNO5ltTnh4NkLtPbz/dRgwG0OqTY+1TcC3ppTQfXh3wNlyh4UHEv6OcRK7kpa2XOfuT54aLgQcW9oJ9HrOTCl9Qtli/+FgUPKu4FvTRcuWMhaxbeHLO9vGtt3axGWkT5+oSP/nIvKHTESiSjdHYoU8iFwLw5ZpsiFlnrZikIXwQuGB/95YoX6yid8w5rkl5/Gnz03vJfr8piPxcFla/KQ/Z99FeFXX3nKPOWr2Tmsu8yb/lKrr6zx9Yea6dZqMgmidA1i26bY3pJ7LW2AspXG7I/umkzxrYkk732/+1BJTJ98YfX6j9w0R3doTt65yxMal97TQOU3JZdG4Ny+g5i70QvoHz9MmTfg0pk+uIPr9V/4KI7uvOoWcxZmDSfnL0puY0hoJQxIi32WlsB5euXIfulBRVJe0u6QdLa9HZyg2MOl/QTSfdJulvSOzP7LpG0XtLq9OfwYj9BPvriD6/Vf+Cim0xirVmEVNaItCLObTc1sALK1y9D9ssc/bUMuMnMlktalj4+ve6Yp4D3mNlaSVOBVZKuN7NN6f7TzOyqAsucu75YY6TVUTplrEvf6wuBldm3kee5DTF6K+fffb8kmSwzqJwAvCG9fylwC3VBxcx+nrn/iKTHgX2BTVREu6M9qvaH1/Follb+gX1d+vDKCNRFGKsGFslFQr8kmSwzqOxnZo+m9x8D9hvrYElHALsBD2Q2f0rSWcBNwDIz25JLSTvUyZKiUf/h1c0tueMPPsQZdxyQ35Kp3cw7aFLmKObDlKlXA3Xso8tS/bDUQ67zVCTdCLy0wa4zgUvNbFLm2N+Y2Q79Kum+/UlqMkvM7LbMtsdIAs2FwANm1rBhWNJSYCnA9OnTX/nQQw91/JnaMW/5yoZNWQOTJnLrsqMKKUMwDeaWbGZ3Tn/mZFY8/9rtDo3m88U6H6ZsvRhoY58H0wNanaeSa03FzI5ptk/SryTtb2aPpgHi8SbHvRj4LnBmLaCkr12r5WyR9K/A34xRjgtJAg9DQ0OFzfbsqU73Bs0LE9nCx3e5khXPbB9Uovl8FWgSKUUv9hv1ag2sgsocUrwCWJLeXwJcU3+ApN2AbwNfqe+QTwMRkgQsAKK7HKnCaI+WJ1o2aUaYqg07bovl81WkScQF0A8j9yqizD6V5cCVkk4GHgIWAkgaAj5gZqek214HTJF0Uvq8k8xsNfB1SfsCAlYDHyi4/OOKvdO9rT6fJh28jzJlu8cxfb6e7ZR2jfViDayCSqupmNkGMzvtu4LyAAAPWElEQVTazGaZ2TFmtjHdPpwGFMzsa2a2q5kdnvlZne47ysxmm9lhZvZuM/tdWZ+lmQVzBzj3HbMZmDQRkfQ1xLSKYlsTLZvMLXnklR+P9vNFP+HOuR7kWYpzFvNoj7b6fJqMxHrVnIXcenyOhexGiNFjzrm2eFDpY21PtKxi80IVy+xchXnurz7mi3I550Lzmkofi3qipXOukjyo9LmY+3y6UeXFkFy19fvfngcV13M6SY/jXAj+t+d9Kq4H9cWaNC5K/rfnQcX1oDGHSpex6qHrGz2VmqlD3vzl2hZ7m3GzodJL9rwdvvMv3a250UgMCRpjKIPrj/WQxuE1lQprOW9X4Pc841v3MLppM8a2NuMi3rtVzYZKf3zXK8KveljW8ryxlaGJMv5Gy+TD9D2oVFZZX+5VaDNulh5nj82PNX5CNwkmy1qeN7YyNFCFC5DQYk/NVARv/qqosb7c8/wDrkqbccOh0rfkkGAyhkzIMZShgbL+RsvWq8P0W+U1lYoq68u9Cun869WaYD7yxNvZzO7b7+w2wWSzgFRkJuQYytBAVS5AXFgeVCqqrC/3qrUZZ5tgrnn+tZz+zMmM2j5YqDU3YsiEHEMZGqjiBYjrngeViirry71qbcb1TTArnn8t87acz2snfCtZZrbbEVIxLA4VQxkaqNoFiAsj1zXqYzQ0NGTDw8NlFyOI2If2xmDmsu/S6C9cwPrlby26OH3H/0Z7RxRr1Lt89XuHYCt83kC5/G+0/5TW/CVpb0k3SFqb3k5uctxzklanPysy22dK+qmkdZKuSNezd2473gTjXLHK7FNZBtxkZrOAm9LHjWzOLCWcXWPw08B5ZnYQ8Bvg5HyL66qoan1AzlVdaX0qktYAbzCzRyXtD9xiZjtcPkr6nZntWbdNwBPAS83sWUlHAmeb2fzx3reX+lRcj/FUKy5iVehT2c/MHk3vPwbs1+S4CZKGgWeB5WZ2NTAF2GRmz6bHjAB+6emqq5ZqJXResqqoeED1AQnb5BpUJN0IvLTBrjOzD8zMJDWrMh1gZqOSDgRWSroHeLLNciwFlgJMnz69nac6V4yxUq1U6Mt1B60Ei7wDas4By9dQ2V6ufSpmdoyZHdbg5xrgV2mzF+nt401eYzS9fRC4BZgLbAAmSaoFxUGgaUIhM7vQzIbMbGjfffcN9vmcCybSVCtdaTXRZZ65ywpItlmFfHhFKrOjfgWwJL2/BLim/gBJkyXtnt7fB5gH3G9JR9DNwIljPd+5yog01UpXWg0WeQbUApJtejqa7ZUZVJYDb5K0FjgmfYykIUkXpcccAgxLuoskiCw3s/vTfacDH5O0jqSP5cuFlt65kCJNtbKddhc4azVY5BlQAwesRqn8PR3N9krrqDezDcDRDbYPA6ek938MzG7y/AeBI/Iso3OFqbXxx9pZ3Um/x14tZoU++qztXxvCBdRWy9CCZn0nf/rKAf5t1eh2TWD9PBfK07Q410SZI3rKHk1U//436K/ZY/OjOx6417Qkh1oj9YEIkmDRKC9ZXp3p7ZRhHPOWr2yYnWEg/f30+uivKgwpdi5aZY7oKXs0UaP3n7D7Y0nCtHpjNSO1UPvaFrz+B1Mnnc9pJwT+Mg5YAxyr78TT0WzjQcW5BspcYKql985xmGyj93/EpjCoX+948HjNSHMWNi1XYcFzjDK0w/PItcZT3zvXQJkjesZ975yHyTZ6/888u5CnrC69Xpf9HlUbiut55FrjQcW5Bsoc0TPue+c8TLbR+694/rV8Zte/DrpmS9WG4noeudZ485dzDZw2/+DtmmaguKvScd8754mSzd7/8Lcuhbl/F+Q9oJrNSd53Mj4PKi5qZY2Cqr1HlO8dcJhsR+8fSJmB2+XHhxS7aNV35ELypdP3TQ4Bh8mWreyh0651PqTYVV6ZI7CiFvtEyTZ4c1Lv8aDiolW1jtxCBRom61xoPvrLRctzKjlXPR5UXLR8XoBz1ePNXy5aZY7Acs51xoOKi5p35DpXLd785ZxzLhgPKs4554LxoOKccy6Y0oKKpL0l3SBpbXo7ucExb5S0OvPztKQF6b5LJK3P7Du8+E/hnCtSo+V8XVzK7KhfBtxkZsslLUsfn549wMxuBg6HJAgB64DvZw45zcyuKqi8znXN05J0ruzFy1xrymz+OgG4NL1/KbBgnONPBK41s6dyLZVzOal9KY5u2oyx7UvRr7ZbU7X1V/pVmUFlPzOrLXr9GLDfOMcvAi6r2/YpSXdLOk/S7sFL6FxAMXwpVrn5yNP2VEOuQUXSjZLubfBzQvY4S1IlN02XLGl/YDZwfWbzGcDLgFcBe1PXdFb3/KWShiUNP/HEE918JOc6VvaXYtVrSp62pxpyDSpmdoyZHdbg5xrgV2mwqAWNx8d4qYXAt81sa+a1H7XEFuBfgSPGKMeFZjZkZkP77rtvmA/nXJvK/lKMoabUDU/bUw1lNn+tAJak95cA14xx7GLqmr4yAUkk/TH35lBG54Ip+0ux7JpSt3w532ooc/TXcuBKSScDD5HURpA0BHzAzE5JH88ApgE/qHv+1yXtCwhYDXygmGI715myc5lVcfneep62J36+8qNzfcJX0nTd8JUfnXPbKbum5PqDBxXn+og3H7m8ee4v55xzwXhQcc45F4wHFeecc8F4UHHOOReMBxXnnHPBeFBxzjkXTN9NfpT0BMkM/iLtA/y64PdsR8zli7ls4OXrRsxlAy9fvQPMbNzkiX0XVMogabiVmahlibl8MZcNvHzdiLls4OXrlDd/OeecC8aDinPOuWA8qBTjwrILMI6Yyxdz2cDL142YywZevo54n4pzzrlgvKbinHMuGA8qzjnngvGgEoikvSXdIGlteju5wTFvlLQ68/O0pAXpvkskrc/sO7zo8qXHPZcpw4rM9pmSfippnaQrJO1WZNkkHS7pJ5Luk3S3pHdm9uVy7iQdK2lN+pmXNdi/e3ou1qXnZkZm3xnp9jWS5ocoT5tl+5ik+9NzdZOkAzL7Gv6OCy7fSZKeyJTjlMy+JenfwlpJS+qfW0DZzsuU6+eSNmX2FXHuLpb0uKSGS6QrcX5a/rslvSKzL9dz1xIz858AP8BngGXp/WXAp8c5fm9gI7BH+vgS4MSyywf8rsn2K4FF6f0vAX9VZNmAPwRmpfenAo8Ck/I6d8DOwAPAgcBuwF3AoXXH/DXwpfT+IuCK9P6h6fG7AzPT19m54LK9MfO39Ve1so31Oy64fCcBFzR47t7Ag+nt5PT+5CLLVnf8h4CLizp36Xu8DngFcG+T/ccB15Ispf4a4KdFnLtWf7ymEs4JwKXp/UuBBeMcfyJwrZk9lWuptmm3fC+QJOAo4KpOnh+ibGb2czNbm95/BHgcGHd2bxeOANaZ2YNm9gxweVrOrGy5rwKOTs/VCcDlZrbFzNYD69LXK6xsZnZz5m/rNmAw4Pt3Xb4xzAduMLONZvYb4Abg2BLLthi4LOD7j8vMfkhywdnMCcBXLHEbMEnS/uR/7lriQSWc/czs0fT+Y8B+4xy/iB3/WD+VVmfPk7R7SeWbIGlY0m21pjlgCrDJzJ5NH48AIZcPbOvcSTqC5Crzgczm0OduAHg487jRZ37hmPTcPElyrlp5bt5lyzqZ5Mq2ptHvOKRWy/en6e/sKknT2nxu3mUjbTKcCazMbM773LWi2WfI+9y1xJcTboOkG4GXNth1ZvaBmZmkpmO106uK2cD1mc1nkHyh7kYy/vx04JwSyneAmY1KOhBYKekeki/LrgQ+d18FlpjZ8+nmrs9dr5L0bmAIeH1m8w6/YzN7oPEr5OY7wGVmtkXS+0lqfEcVXIbxLAKuMrPnMttiOHdR86DSBjM7ptk+Sb+StL+ZPZp+8T0+xkstBL5tZlszr127Ut8i6V+BvymjfGY2mt4+KOkWYC7wbyRV7F3SK/JBYLToskl6MfBd4My02l977a7PXQOjwLTM40afuXbMiKRdgL2ADS0+N++yIekYkqD9ejPbUtve5Hcc8otx3PKZ2YbMw4tI+tVqz31D3XNvKbJsGYuAU7MbCjh3rWj2GfI+dy3x5q9wVgC10RZLgGvGOHaHdtr0y7TWf7EAaDjyI8/ySZpcazqStA8wD7jfkl7Am0n6gZo+P+ey7QZ8m6Qt+aq6fXmcuzuAWUpGve1G8gVTP9onW+4TgZXpuVoBLFIyOmwmMAu4PUCZWi6bpLnAvwDHm9njme0Nf8cBy9Zq+fbPPDwe+Fl6/3rgzWk5JwNvZvsafe5lS8v3MpLO7p9kthVx7lqxAnhPOgrsNcCT6YVV3ueuNUWPDOjVH5K29JuAtcCNwN7p9iHgosxxM0iuKHaqe/5K4B6SL8SvAXsWXT7gj9My3JXenpx5/oEkX4zrgG8CuxdctncDW4HVmZ/D8zx3JKNsfk5yJXpmuu0cki9qgAnpuViXnpsDM889M33eGuAtOfy9jVe2G4FfZc7VivF+xwWX71zgvrQcNwMvyzz3fek5XQe8t+iypY/PBpbXPa+oc3cZyejGrST9IicDHwA+kO4X8Pm0/PcAQ0Wdu1Z+PE2Lc865YLz5yznnXDAeVJxzzgXjQcU551wwHlScc84F40HFOedcMB5UnHPOBeNBxbkeIemDaTp0SyfnOVc4DyrOFSxN6ZKHW4FjgIdyen3nxuW5v5zrgJIFua4DVpGsfXEf8B6SvGNvByYCPwbeb2aW5olaDbwWuEzSz4G/JUmCuQH4czP7laSzSTLjHghMBz5KsmbGW0gyMbzdMjnjsszszrRswT+vc63ymopznTsY+IKZHQL8N8miXReY2avM7DCSwPK2zPG7mdmQmf0j8CPgNWY2l2RNj49njvsDkoy9x5OknbnZzGYDm4G35v2hnOuG11Sc69zDZnZrev9rwIeB9ZI+DuxBsgLffSRp3gGuyDx3ELgiTay4G7A+s+9aM9uaLjuwM0mNCJI8TzPy+CDOheI1Fec6V584z4AvkCxtPBv4fyRJJ2t+n7n/zyS1mtnA++uO2wJgyXoxW21bgr7n8QtBFzkPKs51brqkI9P77yJp0gL4taQ92bZUQCN7sW0djyVjHOdcpXhQca5za4BTJf2MZO2NL5LUTu4lWcfijjGeezbwTUmrgF+HKIykD0saIWlau1vSRSFe17l2eOp75zqQjv7697RD3jmX8pqKc865YLym4lzFSPo2yVyWrNPNrPilY52r40HFOedcMN785ZxzLhgPKs4554LxoOKccy4YDyrOOeeC+f9Ho3FwBQkg2wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Supported set of validities.\n",
    "validities = [0, 1]\n",
    "\n",
    "# Dataset parameters that we will take into account.\n",
    "x_axis = 'param_1'\n",
    "y_axis = 'param_2'\n",
    "\n",
    "# Scatter the data on the plot for each validity class separatelly.\n",
    "for validity in validities:\n",
    "    plt.scatter(\n",
    "        data[x_axis][data['validity'] == validity],\n",
    "        data[y_axis][data['validity'] == validity],\n",
    "        label=validity\n",
    "    )\n",
    "\n",
    "# Plot the data.    \n",
    "plt.xlabel(x_axis)\n",
    "plt.ylabel(y_axis)\n",
    "plt.title('Microchips Tests')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the Data for Training\n",
    "\n",
    "Before we move on with training our logistic model let's extract the data from Pandas data frame and shape training features set and labels set correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the number of training examples.\n",
    "num_examples = data.shape[0]\n",
    "\n",
    "# Extract and shape features.\n",
    "x_train = data[[x_axis, y_axis]].values.reshape((num_examples, 2))\n",
    "\n",
    "# Extract and shape labels.\n",
    "y_train = data['validity'].values.reshape((num_examples, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Init and Train Logistic Regression Model\n",
    "\n",
    "> ☝🏻This is the place where you might want to play with model configuration.\n",
    "\n",
    "- `polynomial_degree` - this parameter will allow you to add additional polynomial features of certain degree. More features - more curved the line will be.\n",
    "- `max_iterations` - this is the maximum number of iterations that gradient descent algorithm will use to find the minimum of a cost function. Low numbers may prevent gradient descent from reaching the minimum. High numbers will make the algorithm work longer without improving its accuracy.\n",
    "- `regularization_param` - parameter that will fight overfitting. The higher the parameter, the simplier is the model will be.\n",
    "- `polynomial_degree` - the degree of additional polynomial features (`x1^2 * x2, x1^2 * x2^2, ...`). This will allow you to curve the predictions.\n",
    "- `sinusoid_degree` - the degree of sinusoid parameter multipliers of additional features (`sin(x), sin(2*x), ...`). This will allow you to curve the predictions by adding sinusoidal component to the prediction curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Theta 0</th>\n",
       "      <th>Theta 1</th>\n",
       "      <th>Theta 2</th>\n",
       "      <th>Theta 3</th>\n",
       "      <th>Theta 4</th>\n",
       "      <th>Theta 5</th>\n",
       "      <th>Theta 6</th>\n",
       "      <th>Theta 7</th>\n",
       "      <th>Theta 8</th>\n",
       "      <th>Theta 9</th>\n",
       "      <th>Theta 10</th>\n",
       "      <th>Theta 11</th>\n",
       "      <th>Theta 12</th>\n",
       "      <th>Theta 13</th>\n",
       "      <th>Theta 14</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>VALID</th>\n",
       "      <td>-3.040355</td>\n",
       "      <td>0.916955</td>\n",
       "      <td>-1.152505</td>\n",
       "      <td>-0.771658</td>\n",
       "      <td>6.043862</td>\n",
       "      <td>2.927590</td>\n",
       "      <td>-8.141133</td>\n",
       "      <td>-13.218318</td>\n",
       "      <td>-22.240853</td>\n",
       "      <td>-10.842197</td>\n",
       "      <td>17.461053</td>\n",
       "      <td>3.658314</td>\n",
       "      <td>35.347278</td>\n",
       "      <td>27.856240</td>\n",
       "      <td>16.066162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>INVALID</th>\n",
       "      <td>3.030662</td>\n",
       "      <td>-0.945259</td>\n",
       "      <td>1.111014</td>\n",
       "      <td>0.816594</td>\n",
       "      <td>-5.970960</td>\n",
       "      <td>-2.873107</td>\n",
       "      <td>8.195278</td>\n",
       "      <td>13.403799</td>\n",
       "      <td>22.486644</td>\n",
       "      <td>10.983047</td>\n",
       "      <td>-17.523614</td>\n",
       "      <td>-3.882551</td>\n",
       "      <td>-35.677803</td>\n",
       "      <td>-28.201402</td>\n",
       "      <td>-16.220361</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Theta 0   Theta 1   Theta 2   Theta 3   Theta 4   Theta 5   Theta 6  \\\n",
       "VALID   -3.040355  0.916955 -1.152505 -0.771658  6.043862  2.927590 -8.141133   \n",
       "INVALID  3.030662 -0.945259  1.111014  0.816594 -5.970960 -2.873107  8.195278   \n",
       "\n",
       "           Theta 7    Theta 8    Theta 9   Theta 10  Theta 11   Theta 12  \\\n",
       "VALID   -13.218318 -22.240853 -10.842197  17.461053  3.658314  35.347278   \n",
       "INVALID  13.403799  22.486644  10.983047 -17.523614 -3.882551 -35.677803   \n",
       "\n",
       "          Theta 13   Theta 14  \n",
       "VALID    27.856240  16.066162  \n",
       "INVALID -28.201402 -16.220361  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set up linear regression parameters.\n",
    "max_iterations = 100000  # Max number of gradient descent iterations.\n",
    "regularization_param = 0  # Helps to fight model overfitting.\n",
    "polynomial_degree = 4  # The degree of additional polynomial features.\n",
    "sinusoid_degree = 0  # The degree of sinusoid parameter multipliers of additional features.\n",
    "\n",
    "# Init logistic regression instance.\n",
    "logistic_regression = LogisticRegression(x_train, y_train, polynomial_degree, sinusoid_degree)\n",
    "\n",
    "# Train logistic regression.\n",
    "(thetas, costs) = logistic_regression.train(regularization_param, max_iterations)\n",
    "\n",
    "# Rename the columns for each theta.\n",
    "columns = []\n",
    "for theta_index in range(0, thetas.shape[1]):\n",
    "    columns.append('Theta ' + str(theta_index));\n",
    "\n",
    "pd.DataFrame(thetas, index=['VALID', 'INVALID'], columns=columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Gradient Descent Progress\n",
    "\n",
    "The plot below illustrates how the cost function value changes over each iteration. You should see it decreasing. \n",
    "\n",
    "In case if cost function value increases it may mean that gradient descent missed the cost function minimum and with each step it goes further away from it.\n",
    "\n",
    "From this plot you may also get an understanding of how many iterations you need to get an optimal value of the cost function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3X+cXXV95/HX+9w7k0lCIJAMSDORBAhaQAQZUpVqrS2Y0gq4ugq6W3hgy6NbqLa6trDtA7u4Vlz3Ye1WWkuFLd0VYotWR0UQFe1DLZDhh2BCAyGoTORHDJDwI5kf9372j/Od5OTm3rlDMmfuTPJ+Ph73Med8z4/7yRHnPd/v+aWIwMzMbCJZpwswM7OZz2FhZmZtOSzMzKwth4WZmbXlsDAzs7YcFmZm1pbDwszM2nJYmJlZWw4LMzNrq9rpAqbK4sWLY9myZZ0uw8xsVrn77rt/HhG97dbbb8Ji2bJlDA4OdroMM7NZRdJPJrOeh6HMzKwth4WZmbXlsDAzs7b2m3MWZmadMjo6ytDQEDt27Oh0KS319PTQ19dHV1fXXm3vsDAz20dDQ0MsWLCAZcuWIanT5ewhItiyZQtDQ0MsX758r/bhYSgzs320Y8cOFi1aNCODAkASixYt2qeeT6lhIWmVpPWSNki6rMU675S0TtJaSTcU2muS7kufgTLrNDPbVzM1KMbta32lDUNJqgBXA2cAQ8AaSQMRsa6wzgrgcuD0iHhG0uGFXWyPiJPLqm/cC889ywOfv5JDT/4tXtH/5rK/zsxsViqzZ7ES2BARGyNiBFgNnNOwzu8CV0fEMwAR8VSJ9TQ1vP0FXjt0Lc9uuHO6v9rMbMrccsstvOIVr+DYY4/lqquumvL9lxkWS4DHCvNDqa3oOOA4Sd+XdIekVYVlPZIGU/u5zb5A0sVpncHNmzfvVZGVSt65inptr7Y3M+u0Wq3GJZdcwte//nXWrVvHjTfeyLp169pv+BJ0+gR3FVgBvAk4H/h7SQvTsqMioh94N/ApScc0bhwR10REf0T09/a2fbRJU0phQTgszGx2uuuuuzj22GM5+uij6e7u5rzzzuPLX/7ylH5HmZfObgKWFub7UlvREHBnRIwCj0p6iDw81kTEJoCI2CjpO8ApwCNTXWSlUskn3LMwsynw37+ylnU/2zal+zz+Fw7mw289oeXyTZs2sXTprl+3fX193Hnn1A6tl9mzWAOskLRcUjdwHtB4VdOXyHsVSFpMPiy1UdKhkuYU2k8HprZPlewchoooY/dmZvuF0noWETEm6VLgVqACXBcRayVdCQxGxEBadqakdUAN+FBEbJH0euDvJNXJA+2q4lVUU0lZnpdyz8LMpsBEPYCyLFmyhMce23WKeGhoiCVLGk8R75tS7+COiJuBmxvarihMB/CB9Cmu8wPgVWXWNm5Xz8JhYWaz02mnncbDDz/Mo48+ypIlS1i9ejU33HBD+w1fggP+cR/jYeFzFmY2W1WrVT796U/zlre8hVqtxkUXXcQJJ0xtD+eADwtlGfWQr4Yys1ntrLPO4qyzzipt/52+dHZGqJG5Z2FmNgGHBVAnc8/CzGwCDgvynoWvhjIza81hwXjPot7pMszMZiyHBVBXhjwMZWbWksMCqCP3LMzMJuCwwMNQZjb7XXTRRRx++OGceOKJpezfYUEeFh6GMrPZ7MILL+SWW24pbf8OC1LPwldDmdks9sY3vpHDDjustP0f8HdwA9SpIA9DmdlU+Ppl8MQDU7vPl70KfmPq3373Urhnga+GMjNrxz0LfILbzKZQh3sAZXHPAvcszMzacVgwfjWUexZmNnudf/75vO51r2P9+vX09fVx7bXXTun+PQwFBBWEw8LMZq8bb7yx1P27ZwHU5Tu4zcwm4rAAgozM5yzMzFpyWAB1+T4LM9s3EdHpEia0r/U5LMh7Fr4aysz2Vk9PD1u2bJmxgRERbNmyhZ6enr3eR6knuCWtAv4KqACfjYg9LkCW9E7gz4EAfhgR707tFwB/llb7HxFxfVl11pX5BLeZ7bW+vj6GhobYvHlzp0tpqaenh76+vr3evrSwkFQBrgbOAIaANZIGImJdYZ0VwOXA6RHxjKTDU/thwIeBfvIQuTtt+0wZtYYq7lmY2V7r6upi+fLlnS6jVGUOQ60ENkTExogYAVYD5zSs87vA1eMhEBFPpfa3ALdFxNNp2W3AqrIKzU9wu2dhZtZKmWGxBHisMD+U2oqOA46T9H1Jd6Rhq8lui6SLJQ1KGtyX7l9+B7fDwsyslU6f4K4CK4A3AecDfy9p4WQ3johrIqI/Ivp7e3v3uohQhQwPQ5mZtVJmWGwClhbm+1Jb0RAwEBGjEfEo8BB5eExm2ykTCDEzr2IwM5sJygyLNcAKScsldQPnAQMN63yJvFeBpMXkw1IbgVuBMyUdKulQ4MzUVorwfRZmZhMq7WqoiBiTdCn5L/kKcF1ErJV0JTAYEQPsCoV1QA34UERsAZD0EfLAAbgyIp4urVb5Dm4zs4mUep9FRNwM3NzQdkVhOoAPpE/jttcB15VZ387vUoXM91mYmbXU6RPcM4NvyjMzm5DDgtSz8DkLM7OWHBakcxa+dNbMrCWHBe5ZmJm147DAJ7jNzNpxWAAoc1iYmU3AYUF+zsJ3cJuZteawAFBGxT0LM7OWHBYAfpCgmdmEHBZAZL4aysxsIg4LAFU8DGVmNgGHBfhqKDOzNhwWpGEoh4WZWUsOC/AwlJlZGw4LgKxCRUHUHRhmZs04LABUASB/vYaZmTVyWABIANRqYx0uxMxsZnJYAGR5z8JhYWbWnMMCUBqGqjsszMyaclhAoWfhR36YmTXjsICdYVF3WJiZNVVqWEhaJWm9pA2SLmuy/EJJmyXdlz6/U1hWK7QPlFknHoYyM5tQtawdKz8RcDVwBjAErJE0EBHrGlb9fERc2mQX2yPi5LLqK5JPcJuZTajMnsVKYENEbIyIEWA1cE6J37f3UlhE3cNQZmbNlBkWS4DHCvNDqa3R2yXdL+kmSUsL7T2SBiXdIencZl8g6eK0zuDmzZv3ulD3LMzMJtbpE9xfAZZFxEnAbcD1hWVHRUQ/8G7gU5KOadw4Iq6JiP6I6O/t7d3rIqT8MLhnYWbWXJlhsQko9hT6UttOEbElIobT7GeBUwvLNqWfG4HvAKeUVunOq6H8bCgzs2bKDIs1wApJyyV1A+cBu13VJOnIwuzZwIOp/VBJc9L0YuB0oPHE+JQZH4aq1z0MZWbWTGlXQ0XEmKRLgVuBCnBdRKyVdCUwGBEDwPsknQ2MAU8DF6bNfxH4O0l18kC7qslVVFPH91mYmU2otLAAiIibgZsb2q4oTF8OXN5kux8AryqztiLtvBrKPQszs2Y6fYJ7RpB7FmZmE3JYAMryDpbv4DYza85hQXEYyj0LM7NmHBYUh6HcszAza8ZhgXsWZmbtOCwAZekO7vBNeWZmzTgs8NVQZmbtOCzYdTWU77MwM2vOYQFkFZ+zMDObiMOCwgluD0OZmTXlsACyShqGCg9DmZk147Cg2LPw1VBmZs04LIBs5yPKPQxlZtaMw4JdJ7jx1VBmZk05LCgMQ/mmPDOzphwWFO+z8DCUmVkzDguKw1AOCzOzZhwWQOaehZnZhBwWQFZJh8FhYWbWlMOCwk15Dgszs6YcFkDFw1BmZhMqNSwkrZK0XtIGSZc1WX6hpM2S7kuf3yksu0DSw+lzQal1jp/gDoeFmVkz1bJ2LKkCXA2cAQwBayQNRMS6hlU/HxGXNmx7GPBhoB8I4O607TNl1FrxMJSZ2YQm1bOQ9H8n09ZgJbAhIjZGxAiwGjhnknW9BbgtIp5OAXEbsGqS275kSmHhE9xmZs1NdhjqhOJM6jWc2mabJcBjhfmh1Nbo7ZLul3STpKUvZVtJF0salDS4efPmdv+GlrL0WlV8B7eZWVMThoWkyyU9B5wkaVv6PAc8BXx5Cr7/K8CyiDiJvPdw/UvZOCKuiYj+iOjv7e3d6yIqvinPzGxCE4ZFRHwsIhYAn4iIg9NnQUQsiojL2+x7E7C0MN+X2or73xIRw2n2s+zqrbTddirtep+Fw8LMrJnJDkN9VdJ8AEn/SdInJR3VZps1wApJyyV1A+cBA8UVJB1ZmD0beDBN3wqcKelQSYcCZ6a2UlR8zsLMbEKTDYu/BV6U9Grgg8AjwD9OtEHkr527lPyX/IPAP0XEWklXSjo7rfY+SWsl/RB4H3Bh2vZp4CPkgbMGuDK1lWJnWPichZlZU5O9dHYsIkLSOcCnI+JaSe9tt1FE3Azc3NB2RWH6cqDpcFZEXAdcN8n69sn4gwTlnoWZWVOTDYvnJF0O/GfgDZIyoKu8sqbfWGQ+Z2Fm1sJkh6HeBQwDF0XEE+QnnD9RWlUdUCeDuoehzMyamVRYpID4HHCIpN8CdkTEhOcsZpsaGXLPwsysqcnewf1O4C7gPwLvBO6U9I4yC5tudTKf4DYza2Gy5yz+FDgtIp4CkNQLfBO4qazCplsd+UGCZmYtTPacRTYeFMmWl7DtrFBX5quhzMxamGzP4hZJtwI3pvl30XBJ7GxXp+KehZlZCxOGhaRjgSMi4kOS/gPwy2nRv5Gf8N5v1HzOwsyspXY9i0+RbpqLiC8CXwSQ9Kq07K2lVjeN6r4aysyspXbnHY6IiAcaG1PbslIq6pA8LNyzMDNrpl1YLJxg2dypLKTT6sp8zsLMrIV2YTEo6XcbG9O7su8up6TOcM/CzKy1ducs/hD4F0nvYVc49APdwNvKLGy6hc9ZmJm1NGFYRMSTwOsl/SpwYmr+WkR8u/TKplld7lmYmbUyqfssIuJ24PaSa+mo8KWzZmYt7Vd3Ye8LXzprZtaawyKpq+JhKDOzFhwWScg9CzOzVhwWSZ0M4Z6FmVkzDoskVCFzz8LMrCmHReKb8szMWis1LCStkrRe0gZJl02w3tslhaT+NL9M0nZJ96XPZ8qsE8bPWTgszMyamez7LF4ySRXgauAMYAhYI2kgItY1rLcAeD9wZ8MuHomIk8uqr1EooxJj0/V1ZmazSpk9i5XAhojYGBEjwGrgnCbrfQT4OLCjxFraCg9DmZm1VGZYLAEeK8wPpbadJL0GWBoRX2uy/XJJ90r6rqQ3NPsCSRdLGpQ0uHnz5n0qNlTx1VBmZi107AS3pAz4JPDBJosfB14eEacAHwBukHRw40oRcU1E9EdEf29v7z7VE8p8NZSZWQtlhsUmYGlhvi+1jVtA/nDC70j6MfBaYEBSf0QMR8QWgIi4G3gEOK7EWvM7uN2zMDNrqsywWAOskLRcUjdwHjAwvjAitkbE4ohYFhHLgDuAsyNiUFJvOkGOpKOBFcDGEmsFZWQ+Z2Fm1lRpV0NFxJikS4FbgQpwXUSslXQlMBgRAxNs/kbgSkmjQB34vYh4uqxaIQ1DuWdhZtZUaWEBEBE3Azc3tF3RYt03Faa/AHyhzNr2+H5Vfc7CzKwF38Gd1LMuqjHa6TLMzGYkh0VSr8yhC4eFmVkzDotxlW66HRZmZk05LJKozKE7RjpdhpnZjOSwGFftoZsxou4roszMGjkskqh2kykYHXXvwsyskcMiUbUHgOEdL3a4EjOzmcdhkagrD4vR4e0drsTMbOZxWCSqzgFgxGFhZrYHh0WSpZ7F2HBHX6thZjYjOSyS8XMWoyPuWZiZNXJYJJXu8Z6Fw8LMrJHDIhkPi9qIh6HMzBo5LJLK+DkLD0OZme3BYZFU58wFoDbqsDAza+SwSMZ7FvXR4Q5XYmY28zgskq45PmdhZtaKwyLpSsNQ9TH3LMzMGjkskvGwiFH3LMzMGjkskq458wCHhZlZM6WGhaRVktZL2iDpsgnWe7ukkNRfaLs8bbde0lvKrBNgTk/qWXgYysxsD9WydiypAlwNnAEMAWskDUTEuob1FgDvB+4stB0PnAecAPwC8E1Jx0VErax6u9NNeTgszMz2UGbPYiWwISI2RsQIsBo4p8l6HwE+DhTHf84BVkfEcEQ8CmxI+ytNVqkwElWHhZlZE2WGxRLgscL8UGrbSdJrgKUR8bWXum0ZhulCNZ+zMDNr1LET3JIy4JPAB/dhHxdLGpQ0uHnz5n2uabvmko2+sM/7MTPb35QZFpuApYX5vtQ2bgFwIvAdST8GXgsMpJPc7bYFICKuiYj+iOjv7e3d54J3ZHOpOCzMzPZQZlisAVZIWi6pm/yE9cD4wojYGhGLI2JZRCwD7gDOjojBtN55kuZIWg6sAO4qsVYAhrN5dI05LMzMGpV2NVREjEm6FLgVqADXRcRaSVcCgxExMMG2ayX9E7AOGAMuKfNKqHEj2Ty6ag4LM7NGpYUFQETcDNzc0HZFi3Xf1DD/UeCjpRXXxEh1Pofs+Nl0fqWZ2azgO7gLatX59IQfUW5m1shhUVDrms/ceLHTZZiZzTgOi4J690HMd8/CzGwPDouiOQvo1hgjw74xz8ysyGFRoDkLAHjxuWc7XImZ2czisCjIehwWZmbNOCwKqnMPBmDHC1s7XImZ2czisCjoSmEx4rAwM9uNw6JgzoJFAGzf+lSHKzEzm1kcFgVHLDsegOEnH+pwJWZmM4vDouCQw3r5OQvJnt7Q6VLMzGYUh0WDp7qXsuD5RztdhpnZjOKwaPDcQcs5cvSnnS7DzGxGcVg0qC9cxkKe5/ltz3S6FDOzGcNh0aBrYf6q7y2P/6TDlZiZzRwOiwZzF+Vvc33uKYeFmdk4h0WDg494OQDbtwx1uBIzs5nDYdFg8ZHLABjbuqmzhZiZzSAOiwZz5y9gK/PJnnu806WYmc0YDosmns4W0/3iE50uw8xsxnBYNLFtzss4eNg9CzOzcaWGhaRVktZL2iDpsibLf0/SA5Luk/Q9Scen9mWStqf2+yR9psw6G+2Y38fhY08Q9fp0fq2Z2YxVLWvHkirA1cAZwBCwRtJARKwrrHZDRHwmrX828ElgVVr2SEScXFZ9E4lDj2LBz7ez9ZnNHLLoiE6UYGY2o5TZs1gJbIiIjRExAqwGzimuEBHbCrPzgSixnknrXnw0AE895qfPmplBuWGxBHisMD+U2nYj6RJJjwD/E3hfYdFySfdK+q6kN5RY5x4OOfIYALb9zE+fNTODGXCCOyKujohjgD8B/iw1Pw68PCJOAT4A3CDp4MZtJV0saVDS4ObNm6espiOPPoGtzOdlaz7GE485MMzMygyLTcDSwnxfamtlNXAuQEQMR8SWNH038AhwXOMGEXFNRPRHRH9vb++UFT7voEN48uwbWBDP0XPtr7Dmy38zZfs2M5uNygyLNcAKScsldQPnAQPFFSStKMz+JvBwau9NJ8iRdDSwAthYYq17OO41b2LLOwf4WdcyTr3nv3HXFz7lq6PM7IBVWlhExBhwKXAr8CDwTxGxVtKV6congEslrZV0H/lw0wWp/Y3A/an9JuD3IuLpsmptZfnxp3H0H93K2p6TWfnAh3nko/3c/bXPUhsbm+5SzMw6ShEz4gKkfdbf3x+Dg4Ol7HtkeAf3feVvOHLt37M0fsb66itZ9Ds3sfhlS9tvbGY2g0m6OyL6263X8RPcs0H3nB5WvuMDLPmzH7Hm5L/gqNFH6P7MSh5de2enSzMzmxYOi5cgq1Q47dxLePL8bwDi2a/9eadLMjObFg6LvXDUK1/D2pe/h1Ne/AGP3P+DTpdjZlY6h8VeOv5tf8I25nHwF9/NHZ/5fZ79uZ9Sa2b7L4fFXjrk0MX89Ixr2DTvlZz6+Grqnz6NH97+z50uy8ysFA6LfXDi6W/l5D++haF3fp1nKos54TsXc+dfX+BLa81sv+OwmALLT/glev/gm9y78Ax+acuXuOv6y3wDn5ntV3yfxRSKep3Bv3oXp239BiNRYasO5vnsEJ7r7uWFQ3+R7r5X07uin0VHLmfe/AUoc1abWWdN9j4Lh8UUq9dqDA78DbXND1HZvoWu4Wc4ePhxlo79lG7Vdq5XC/Hj6nKePegYoms+9a751KrzoNJFZN2o0kVUulC1G7IuVOlC1S5UmYMqXdA9D1XnkFWrZFmVSrWLrFKhWukmq1SoVKv5p5K3K6uSVaooq0ClQlbpIssqZAJJZIJqltHTlSGpg0fQzKbTZMOitJcfHaiySoWVb/uDPdpHhnfwyPp7eHrjPdS2PQnDWzn4yUGO2PoAc9nOvNjOXI1Me71jkVGjQo2MUTKGyQhVqFOhrnxZnWzndOzxc3x53lZXhaAwrQpkFUg/lVWpVitUq1WqlWq+LMvy5ePrZhWkSh5sWf6zXu2h3n0IMedg6unDnIOh0kWWZVSyDCmjUqmQZaKSZRxxyDx6uiqgDCRAzaeVpfn0MbM9OCymSfecHo456fUcc9LrW65Tr9UZGxuhNjrM6NgI9ZFhxkaHqY+NUhsdpjY2QoyNUBvdQYzuoD66g6iNUqvVqNdq1GujRL1G1Eap12tEvUa9Ngb1GqqPQdShXoMYQ/UaihpEDdXzn1GvMTIywujoGIp8fUWNLPJ18+n6rmnqZDGGok4WNSoxUmjf9VNRp8KubYg6WR5HKYqiMJ1+qnM93tgZHOMh0ipsxj/jgZihwnQxJPPtCsG4x7KssM14eGZp/ayhrfAz64KsCpWu3eezKlTSz2x8WbXhe3eF867pKvQshPmLoXt+Pr/zu6ppvWpelx1QHBYzSFbJ6K70wJwe5na6mBJFBNtHazw/PMZIHeoR1Oqx+89aUKuNEfUxarUaMbqdbHgrGt6GdmwjG9lGNryVqI9BPahHjYiACOr1OvV6nc3btvPTp1/giWe3E1FHeQyQEYgAYud0RiDly0U9/dy1fNcHssJypaDLdgZdUFFjGO4KxIw6FdWoMLrbsuLPCvXCftMy7bmsQp0qNaqpbzg+XdX0XFwxFhk15b3S9KcBNVLAjv9vze49tdhturEX13y7xj8bdt9usvtvXK/1/nerQ5Ncb4I6xtfbpgX8wdyPtdhDmm/o2e7Rz1XzZb945MF8+t2v2aPCqeSwsGkniXndVeZ1T89/fttHavz7E9vY8vwIz7w4Qj2CerDzZ0RQr6dp0nzDOuPLi6LhV0Lx9N8ev1gazg3u8Qtwgn3vuazNtlFH1KjEWN7Dq9d29gDz3l1tZ29wV48x/3Wv+hg9Y88xf+wZqvXhfB3yHmVWWFfFXiW7epxZ1GgpdoWY9vh37JpTRNP2PeZbnm8d/2OgxS4avqu4WEz03buaxteLJpFXrKu4vx3ZPE45cmHLvbf/3zlaLjtq0bw9a51iDgvb783trnDKyw/tdBlmvLbTBewDDzyamVlbDgszM2vLYWFmZm05LMzMrC2HhZmZteWwMDOzthwWZmbWlsPCzMza2m+eOitpM/CTfdjFYuDnU1TO/sbHZmI+Pq352LQ2U47NURHR226l/SYs9pWkwck8pvdA5GMzMR+f1nxsWpttx8bDUGZm1pbDwszM2nJY7HJNpwuYwXxsJubj05qPTWuz6tj4nIWZmbXlnoWZmbV1wIeFpFWS1kvaIOmyTtfTCZKuk/SUpB8V2g6TdJukh9PPQ1O7JP3vdLzul1Tu67k6TNJSSbdLWidpraT3p/YD/vhI6pF0l6QfpmPz31P7ckl3pmPweUndqX1Omt+Qli/rZP3TRVJF0r2SvprmZ+XxOaDDQlIFuBr4DeB44HxJx3e2qo74B2BVQ9tlwLciYgXwrTQP+bFakT4XA387TTV2yhjwwYg4nvzdNZek/0Z8fGAYeHNEvBo4GVgl6bXAx4G/jIhjgWeA96b13ws8k9r/Mq13IHg/8GBhflYenwM6LICVwIaI2BgRI8Bq4JwO1zTtIuJfgacbms8Brk/T1wPnFtr/MXJ3AAslHTk9lU6/iHg8Iu5J08+R/59+CT4+pH/j82m2K30CeDNwU2pvPDbjx+wm4NfU+NLp/YykPuA3gc+meTFLj8+BHhZLgMcK80OpzeCIiHg8TT8BHJGmD9hjloYFTgHuxMcH2DnEch/wFHAb8AjwbESMpVWK//6dxyYt3wosmt6Kp92ngD8Gxl9AvohZenwO9LCwSYj8krkD+rI5SQcBXwD+MCK2FZcdyMcnImoRcTLQR95Tf2WHS5oxJP0W8FRE3N3pWqbCgR4Wm4Clhfm+1Gbw5PjwSfr5VGo/4I6ZpC7yoPhcRHwxNfv4FETEs8DtwOvIh96qaVHx37/z2KTlhwBbprnU6XQ6cLakH5MPcb8Z+Ctm6fE50MNiDbAiXZ3QDZwHDHS4ppliALggTV8AfLnQ/tvpqp/XAlsLwzH7nTRmfC3wYER8srDogD8+knolLUzTc4EzyM/p3A68I63WeGzGj9k7gG/HfnyjV0RcHhF9EbGM/HfLtyPiPczW4xMRB/QHOAt4iHys9U87XU+HjsGNwOPAKPkY6nvJx0q/BTwMfBM4LK0r8ivIHgEeAPo7XX/Jx+aXyYeY7gfuS5+zfHwC4CTg3nRsfgRckdqPBu4CNgD/DMxJ7T1pfkNafnSn/w3TeKzeBHx1Nh8f38FtZmZtHejDUGZmNgkOCzMza8thYWZmbTkszMysLYeFmZm15bCw/ZKkIyTdIGmjpLsl/Zukt+3jPv9c0n9N01dK+vW93M/Jks5qsWyepM9JekDSjyR9T9JBkhZK+v19qd9sXzgsbL+TbqT7EvCvEXF0RJxKflNUX5N1q41tkxERV0TEN/eyxJPJ79Vo5v3AkxHxqog4kfyel1FgIeCwsI5xWNj+6M3ASER8ZrwhIn4SEX8NIOlCSQOSvg18K/3l/i1J96S/6Hc+eVjSn0p6SNL3gFcU2v9B0jvS9KmSvpt6MLcWHgPyHUkfT+98eEjSG9KTAq4E3iXpPknvaqj9SAqPB4mI9RExDFwFHJO2+UTa/4ckrVH+3ozxd0ksk/TvqXfyoKSbJM1Ly65S/l6O+yX9ryk72nZA2Ku/qsxmuBOAe9qs8xrgpIh4OvUu3hYR2yQtBu6QNJDWOY+8J1BN+9ztoXDpuVF/DZwTEZvTL/+PAhelVaoRsTINO304In5d0hXkd3Zf2qSu64BvpCD6FnB9RDxM/r6MEyN/aB+SziR/Z8ZK8rvGByS9Efgpeai9NyK+L+k64Pcl/R/gbcArIyLGH9NhNlkOC9vvSbqa/LEdIxFxWmq+LSJcgT9DAAAB/ElEQVTG3+Eh4C/SL9s6+aOijwDeAPxLRLyY9tPsuWGvAE4EbkuvHqiQPzpl3PiDB+8GlrWrNSLuk3Q0cCbw68AaSa8Dtjesemb63JvmDyIPj58Cj0XE91P7/wPeR/6o7B3Atcrf2PbVdrWYFTksbH+0Fnj7+ExEXJJ6DIOFdV4oTL8H6AVOjYjR9JTQnkl+l4C1EfG6FsuH088ak/z/W+QvFPoi8EVJdfLzG19o8r0fi4i/260xf+dG4zN8IiLGJK0Efo38IXWXkg/XmU2Kz1nY/ujbQI+k/1JomzfB+oeQv3dgVNKvAkel9n8FzpU0V9IC4K1Ntl0P9Ka//pHUJemENvU9ByxotkDS6dr1Pu9u8tf9/qTJNrcCFyl/zwaSlkg6PC17+Xg9wLuB76X1DomIm4E/Al7dpkaz3TgsbL8T+dMxzwV+RdKjku4if13ln7TY5HNAv6QHgN8G/j3t5x7g88APga+TP9K+8btGyP9S/7ikH5I/lfb1bUq8HTi+xQnuY4DvplruJe8NfSEitgDfT5fTfiIivgHcAPxbWvcmdoXJevJ3hT8IHEr+HvAFwFcl3Q98D/hAmxrNduOnzprtR9Iw1FfTZbdmU8Y9CzMza8s9CzMza8s9CzMza8thYWZmbTkszMysLYeFmZm15bAwM7O2HBZmZtbW/wcqeAzGf9IbPgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Draw gradient descent progress for each label.\n",
    "labels = logistic_regression.unique_labels\n",
    "\n",
    "plt.plot(range(len(costs[0])), costs[0], label=labels[0])\n",
    "plt.plot(range(len(costs[1])), costs[1], label=labels[1])\n",
    "\n",
    "plt.xlabel('Gradient Steps')\n",
    "plt.ylabel('Cost')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Model Training Precision\n",
    "\n",
    "Calculate how many flowers from the training set have been guessed correctly. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Precision: 84.7458%\n"
     ]
    }
   ],
   "source": [
    "# Make training set predictions.\n",
    "y_train_predictions = logistic_regression.predict(x_train)\n",
    "\n",
    "# Check what percentage of them are actually correct.\n",
    "precision = np.sum(y_train_predictions == y_train) / y_train.shape[0] * 100\n",
    "\n",
    "print('Training Precision: {:5.4f}%'.format(precision))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Draw Decision Boundaries\n",
    "\n",
    "Let's build our decision boundaries. These are the lines that distinguish classes from each other. This will give us a pretty clear overview of how successfull our training process was. You should see clear distinguishment of three sectors on the data plain. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEXCAYAAABlI9noAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzsnXd8FNX2wL9nN51AEpJQUoDQmwgSsaCCgIKCiIoI+igqoj57R997dp9Ynu1n7x0EK1iwIfYCSBOkB0JCKAlJgPTs3t8fu4EQNmV3Z3dnN/f7+SzZmblz5+yyM+eecs8VpRQajUaj0RiBJdACaDQajSZ00EpFo9FoNIahlYpGo9FoDEMrFY1Go9EYhlYqGo1GozEMrVQ0Go1GYxhaqWiaFSLyvIj8x8/XVCLStZ5jF4nIV/6UR6PxJaLnqWhCARHZCqQAKUqp/Fr7lwP9gQyl1NYAyaaAbkqpTX641snAFzWbQAxQUqtJb6VUtgf9RgFlQLpSKsdrQTUhi7ZUNKFEFjCpZkNEjsLxUPUKcRAU94pS6kelVKxSKhbo49wdX7PPE4Wi0bhDUNwoGk0TeQuYUmt7KvBm7QYi8rqI3F9r+2wRWSEi+0Rks4iMcu5fLCIPiMjPQCnQWURSRGS+iOwVkU0iclmtfqwicoezj/0iskxE0mtdeoSIbBSRIhF5RkTEed40EfmpVj9KRK4VkS0iki8ij9QoNBHpKiLfi0ix89h7nnxJItJaRN4UkZ0isl1E7qp1jZ4i8pPzGntEpOb7+8H5d72IHBCRcSLSTkQWOj9TgYgs8kQeTWgRFmgBNBoD+Q2YLCK9gA3ARGAwcL+rxiIyCIfSGQ98C7QHWtZqMhk4A1iPw5X0LfAXDjdbT+BrEdmslFoE3IjDSjrTee1+OJRRDWOAY4FWwDJgAbCwns9xDpAJxALfOK//MnAf8BVwKhDhbOMJ7wCbgM5AHPA5sBV4A3gQ+Bg4GYgCjnGecwoO91ePGveXiDzulG0MjgHq8R7KowkhtKWiCTVqrJXTgL+B3AbaXgq8qpT6WillV0rlKqXW1Tr+ulJqjVKqGmiHQ0HdppQqV0qtwPGgr7GMpgP/VkqtVw5WKqUKavU1SylV5HQ/fYcjzlMfDyml9jrbPsEhl14V0BFH3KhcKfVTvT3Ug4h0xKEgblRKlSql8oCncCjgmmt0AtoppcqUUj830F0VDgXbQSlVqZT6oYG2mmaCViqaUOMt4EJgGnVcXy5IBzY3cHx7rfcpwF6l1P5a+7YBqU3sa2et96U4rJCmXHeb89oAt+KwmP4QkTUickkDfdRHRxwWyB6n26oIeBJo6zx+A4441HIRWSUi/2igrweAHcB3TnfgjR7IowkxtPtLE1IopbaJSBYON9SljTTfDnRpqLta73cArUWkZS3F0oFDllBNX3+5L/URpANral1jB4BSaidwGYCInAR8IyI/uJlVth04ACQoF6mfSqlc4BJnzGcI8JWI/ADsctG2GLgOuE5EjsahXH5vxLrRhDjaUtGEIpcCw5RSJY20ewW4WESGi4hFRFJFpKerhkqp7cAvwIMiEiUi/ZzXedvZ5GXgPhHp5swW6yciiR7Kf4uIJDgD/dcB7wGIyPkikuZsU4hD6dnd6VgplYUj9vSwiLR0fu5uTiWFiFwgIilOhVPkPM2mlKoAinHEYXC2HSsinZ0KqBiwuSuPJvTQSkUTciilNiulljah3R/AxcDjOB6K3+NwD9XHJBzxhh3AR8BdSqlvnMceA+biCKTvw6Gwoj38CJ/gCOavAD5z9gWOQP/vInIAmA9cp5Ta4kH/k4B4YB2wF4fSqnF/nQAsc15jHjDDab0A3AnMc7rNxgK9cMSH9uPIDntUKfWrB/JoQgg9+VGjMRH+nCip0fgCbaloNBqNxjC0UtFoNBqNYWj3l0aj0WgMQ1sqGo1GozGMZjdPJSkpSXXq1CnQYmg0Gk1QsWzZsnylVHJj7ZqdUunUqRNLlzaabarRaDSaWojItqa00+4vjUaj0RiGVioajUajMQytVDQajUZjGM0upqLRaDSBoKqqipycHMrLywMtSoNERUWRlpZGeHi4R+drpaLRaDR+ICcnh5YtW9KpUyecC3+aDqUUBQUF5OTkkJGR4VEf2v2l0Wg0fqC8vJzExETTKhQAESExMdEra0orFY1Go/ETZlYoNXgro3Z/aZo9Hy/P5ZEv17OjqIyU+GhuGdmDcQNSGz9Ro9EcgbZUNM2aj5fncvuHq8ktKkMBuUVl3P7haj5e3tDS9hpN8LJw4UJ69OhB165dmTVrluH9a6WiadY88uV6yqpsh+0rq7LxyJfrAySRb/l4eS6DZy0iY+ZnDJ61SCvPZobNZuOqq67iiy++YO3atcyePZu1a9caeg3t/tI0a3YUlbm1P5ipscpqlGiNVQZod58J8YVb9o8//qBr16507uxYFXrixIl88skn9O7d2wiRAW2paJo5KfGuV/ytb38w09yssmDGV27Z3Nxc0tPTD26npaWRm2ustaqViqZZc8vIHkSHWw/bFx1u5ZaRPQIkke9oTlZZsBPMAwCtVDTNmnEDUnnw3KNIjY9GgNT4aB4896iQdAc1J6ss2PHVACA1NZXt27cf3M7JySE11djfuo6paJo94wakhqQSqcstI3scFlOB0LXKgp2U+GhyXSgQbwcAxx57LBs3biQrK4vU1FTmzJnDu+++61WfddGWikbTTGhOVlmw4yu3bFhYGE8//TQjR46kV69eTJgwgT59+njV5xHXMLQ3jUZjapqLVRbs1Pwf+WJS7plnnsmZZ57pdT/1oZWKRqPRmJBgHQAE3P0lIq+KyG4R+aue4yIiT4nIJhFZJSLH1Do2VUQ2Ol9T/Sd1aKEnxLmH/r40mvoxg6XyOvA08GY9x88AujlfxwHPAceJSGvgLiATUMAyEZmvlCr0ucQhRHOcEOfNpLLm+H1pNO4QcEtFKfUDsLeBJmcDbyoHvwHxItIeGAl8rZTa61QkXwOjfC9xaBHM+fCe4O2ksub2fWk07hJwpdIEUoHttbZznPvq238EIjJDRJaKyNI9e/b4TNBgpLlNiPNWKTS370ujcZdgUCpeo5R6USmVqZTKTE5ODrQ4psJ0E+JWzYXH+8Ld8Y6/q+Ya2r23SsF035dGYzKCQankAum1ttOc++rbr3EDU5UpWTUXFlwLxdsB5fi74FpDFYu3SsFU35dG4yaXXHIJbdq0oW/fvj67RjAolfnAFGcW2PFAsVIqD/gSOF1EEkQkATjduU/jBqaaEPftvVBVx2KoKnPsNwhvlYKpvi+Nxk2mTZvGwoULfXqNgGd/ichsYCiQJCI5ODK6wgGUUs8DnwNnApuAUuBi57G9InIfsMTZ1b1KqYYC/pp6ME0+fHGOe/s9wIhJZYH6vvQKlc2MVXMdA6riHIhLg+F3Qr8JXnV5yimnsHXrVmPkq4eAKxWl1KRGjivgqnqOvQq86gu5NO5hyAMvLs3p+nKx30D8pRSMVAI6lbmZUeMKrrHca1zB4LVi8TXB4P7SmBzD1n4YfieE14lthEc79gcZRq+HoVOZmxl+cAX7Cq1UmilGzgo37IHXbwKc9RTEpQPi+HvWU6YfmbnCaCWgU5mbGX5wBfuKgLu/NP7HaFeKoQ+8fhOCUonUxWgl4KtS6BqT4idXsC/QlkozxOhRtJ67cSRGfyc6lblxQqomm49cwZMmTeKEE05g/fr1pKWl8corr3jVnyu0pdIMMXoUrRd/OhKjvxNflkIPBUIukaHGWjc4+2v27NkGCNcwWqk0Q4x2pegH3pH44jsxTeq3CWnI+g7a7yxIXcFaqTRDfGFZ6AfekejvxH/oRAbzoGMqzRA9K1wTagRLXM8x7c7ceCujtlSaKXoUrQklgiGuFxUVRUFBAYmJiYhIoMVxiVKKgoICoqKiPO5DKxWNxgXVVdW89MDHZG0z91IJ7drE8c+7ziUqxvOHgLuYsVxMMMT10tLSyMnJwezLb0RFRZGW5nnqsgSDOWYkmZmZaunSpYEWQxNAtm/IpXD3vnqP78kr5D+fLCJhwm4SWpT4UTL3OVAexY6P2vHvEwfTqXv7g/vjEmPp2Cu9gTM9o26WFTgsAu0+DX1EZJlSKrOxdtpS0TQb9hce4KorX2B1/1IsrarrbRcdU8XpN60mI7bAj9J5Tu618Ty2ej/7Vx2KH9hKwuh5ZyTPPXs5Cclxhl0rJLOsNIailYom6FFK8cpDn/DyqjVURwgo+6GDcigXpTpJcexV6/lnmywiLDYXPR3CpoTwzXDm2k1IbWu+ZXuYtsDoj+AhikV571Nt/5iLB/102JEqu4VlJ3Zk2P/+D+vOQ99BWKXisv59ueSWsR759XWWlaYxtFLRmJp1yzbz1CPzqaiy19tmG2WEXVjA6DM2NKgsoqyVxIeXs6+yLe1aDaa+R6pgYeCHj9Ku0IXra382RHR081P4jnM63kxB+fn8vucj7DisLwVs3fc7x7fJoteUPMpsEQfbV9qtzNuUz+zxq+lINNERYdz8n/Po1LtpPnRdLkbTGFqpaALHqrlULLyPX/+0URWVBL3HQfpxBw9/9d1qFvfI46grtxJmrV+pDIwqoWernVTZrVSUhR/ZQAQiWiBEkRozmSldpzU+Sre/CrhQKgbVXjIy2J0Y1ZHK/PMP6+/mkRNp3fpHVhXOJcZ6SNEmhBdxwVG/s+72dhSWt+CAzcp5HxVx1nPpDD6xB9EtIjnhzAFYw6wurxUMWVaawKID9ZrAsGouix6ZxcywsUQMqcTiQmmktivglPQNxFirGu2uvLofUz+cS1SVK0tF4O4it+U7bD0LcNReMqBqstHBbnf6K68u5o2NtxEVturgvgPVEXy3rSe798RTVWXF+k00z1w1nqMH96z3embOstL4hqYG6rVS0fiMwl1FXH71C2xItqNcGAatBu7jjMyVpEbX/8DfV9mSM9LuIi22T71trBJGuKUFPN63nsqu6XDDX+5/AB+svAcweNYily6k1Phofp45zC/9VdlLsKlqNhcvY/HO/xIbXnrw2OYDSXz5Y3/K10djqVYMrmjJo89eZmjaslZMwUfQZH+JyCjgScAKvKyUmlXn+OPAqc7NGKCNUireecwGrHYey1ZKjfWP1Jra1ATKP//1b2qGKArY3r2a465fx4nxu0COHLzEWisQIGqLnfTiYudegaG3AxBtbcnRrcdgkSb+TIff6dq68LSyq49qLxkd7Pakv3BLC8KBPq2H0T3uRFYVfkqFvYyNxSvIaPEbl478jtIREdiVheW7OnDKNY8ya8xpDDvnuHr7bCohV/xRcxgBVSoiYgWeAU4DcoAlIjJfKbW2po1S6oZa7a8BBtTqokwp1d9f8jZntqzZzi9frDhiv81m550lq7FemE/3U/MOOzYktoDWEaXsq0zEQsvDTyzZjb3cyrl/rCNpf/mh/XHp0GayZ0L6qLKr0Rgd7Pa2v3BrFAOTxgNwYpvJbC9ZzWfbHyOcCpB8RqT/Ta/r8rhzSQmvTviRUYN6EB4exhmTT6ZV61i35dVpyaFNoC2VQcAmpdQWABGZA5wNrK2n/STgLj/J1iyx2+2UFB9yhSi74sH/zObrjnkkdXXhphLF0cO20zsuD0sdF1dZdSRdW13P4LYuHuqr5sJX10JVLYVixNLBQVDZ1ehgt9H9pbc4iit6vgaAXdmYn/0k1fYPmXbSj/zRPYO3d+diV8LD9/7JrT0zueCKkW71b6Slpt1o5iPQSiUVqO0EzwFc2tci0hHIABbV2h0lIkuBamCWUurjes6dAcwA6NChgwFihyY/fraMW2d/QUnHWnVGrdBr3FYu7bC+wYB5hW0AU7s9RKS1iSPXILEqfIHRJUV8WaLEIlbGdbyR/LJzmb3lJga328zgdpsByO/Zghf/KuSd8/7k5Scvo01aUpP6NMpS0240cxLQQL2IjAdGKaWmO7cnA8cppa520fY2IE0pdU2tfalKqVwR6YxD2QxXSm1u6JrNKVDf2CjuQHEJt/zzZTbtP4BNoGJMKSOPXUly1P6DbQSItlaxr7IVx7W5mBZhrQD4M7uIL/7aSVFpJVZbey49bri+kUMcpRRbD/xOaXUh2Qc2s71kLmEWGyv2pvPTgj7MiOnD5Xec02i6tlHZb0YnPGgaJlgC9blA7QJFac59rpgIXFV7h1Iq1/l3i4gsxhFvaVCpNBcOu3GVYt/va5n15e9807E1HZNiKC2vZL7kcdRlmxjYshiLKDJaFGBBUa26Y5VDE+ZaR3ZnctdrDwbMP16ey38/XE1ZVauDbW7PNcEI0UfZWkEvi0GICBktjwegTwKUVE/k9Q03MTBxI10u2sP8dfl8OH4tL94/jU696p/PY5RlpWf3m5NAWyphwAZgOA5lsgS4UCm1pk67nsBCIEM5BRaRBKBUKVUhIknAr8DZtYP8rmgulsrguxeye0chUllF6x2biZhcTlLSoSKKVoudQalZtIk8QJXdil1ZqLC1YWLnh0iO7txw34EcIdb3sPbhvBKPZDSLLE3BSwW4ZM98luY/TpS1grXF7Vn0VT/OK+rMbbMuxGp1PYnSCLSl4l+CwlJRSlWLyNXAlzhSil9VSq0RkXuBpUqp+c6mE4E56nAN2At4QUTsOBYbm9WYQjEbvggy2u12nrxrLgcObCG6iyIsqprjr/yboxNysdRJ662whZEaM53hKRe7VQcqYCPEug/r4u2ObXA8FKvqXL+qzLG/vgekr6wJT2QJFA19p02U9djksfRrPYw3Nt5G3/jldDq3gG+27ObUi7bx7A0X0Pe4bj4RXc/uNyd68mOA8EUJ8Y0rt3L5/W8TO3kXw7uuJTasHAGsoli3K51f/zyJ1tEx/N9FxwCQGNGZqDD3K9gGbITY0OTG4hzA1W+5ntn0vrQm7o53T5ZAYvCE0bUrZ/Gd9UtiIyvYuD+Zr37oz5C/2/HAU9OJiHRRQsdLdPaX/wgKS6U5Y0SuflVlFS/c/xHbsvPZW1rOqsx9DL9tJb3idlFSFcWq7F5U24WNW3qwYWsvosOtXH/uUaTGeHfTBWyEWJxT//64tHoejvX49n1pTbgrSyBp6Dt1l1Vz6f3pE3SzlzP7+L506biHS0Z9x/d9enDK5Q/z2KTRnDjS2GllZljBVCu2w9Fr1AcIb11ISxb9xSmXPsyXmT+yc8oyuHoFl4z9jp6tdlFtH8T0Hgs4NfEeVvw5iY1bexm6Dn3A1riv76Fc47oKr5OS2tC8FyMfpnVxV5ZA0tB36i5ORR1uU0z5eTWnfpVNVWUYozqtYfQNv3DL1nlcdtETlO4vbbyvIKHG45BbVIbiUFrzx8vryzcKfbSlEiA8zdUvKynnxn++wIrj9zDqphV0arH34LH9VbEMbv8w3eIcU33GDWjhswd9QEaIDZVh6TcBsn+DZa+DsoFY4egL67c6fGlNBNMcHCNL29RRyJ3y93PVR8v4MLMnKV2LmDb0B37p0YUhtz7G3ScNYfRFJ3spfNPwpSWhqwMciVYqAcITF9LX837lX19/Q69pW7g4ZRNhYifSOpIz0qYiWIkNb9/0Oll1CYYU2IYe1qvmwsp3HQoFHH9Xvgsdjnf9OYyuE+ZKVrN9f64wUgG6UNQWBeM3lLBr2Me8t+UmhqZuoOc/d/Lw8v28df6vhq9MWRdfT5DUac1HogP1AaSpI6h9ew9w5ZXPs31UPmcMXElKdDHFlYmc3+kR2rdwXZ7cLYItBdYVngScg0GRBhON/I6UsvNFzvPkls7GIoqlezryx0e9uKHDQC66ZpRHK1E2hq+TSppTWrMufV8PZlIqTeG957/k4XVLGTh+HYOSswBoG30+Y9KvQcSgkJjRJeMDQTBlXAUYnwaWm6Coiyt38ObGG2gVsZ2CyhYs/Ksf4fMSeeV/l9GuY7IxcjjJmPlZfb8KsmaN9rp/X2RxmhWd/RXk7M7JZ/q1L1EyoYALp68kKbKE4spUJnd9nIRIg7OIfBm09hfBlHEVQHxeL6sJbr+4iBSu7j2HH3bOobz4eSYN+I2VnVIZ/fJzTLP24uq7xhtmtfh6+WNf1l0LVrRSMRlKKV7470e8VLKGwbeuoX/r7VTbrXRocQVDu04+dLMZ6boJhQeyr2MkIYJZAssiwpD2k8hMHskbG25mQOt1tB5fyvMfRmK7+S2u/9+UBs9vqrXlj/R3M6Q1mwmdUmwilFJMPvO/vNxmJVP+sZiBidmUVXfloi4fcmrKlMMVyoJrnYpAHZoFvWquZxcOphTY+ug3weG7j0sHxPE3mGJCfsJVAHms5SfeK73M4UJ8vK/nvyMPaBHWmn/2fhXoSFLEASLSSvll+VZsNlfLQjtwJ403YOnvzRgdUzERBXmFjLniWVKv3cZZHf6iU+xFDGl/1ZENfRED0UFrc+Dj/4e6geWxlp+YFf4yMVJ5qFEAkjR+3f0mG4qf50B1BN9s6UPBW+3rLfHSnILjZqKpMRVtqZiEv5du5rzrn6HNZXkMTdmIUnBM0jjXjX0RA+k3waGQ7i5y/NUKxf8YbYG64JaRPYgOP1Tk8dawuYcrFDhUWcCPnNBmCscmzQIVxrjuyznl1iVc8uvb3DzjOSorDl/HR6fxmhutVExAyb5Srrj1TaKm5HNujz+xAgOT76NleD0mupGzoJszq+Y6rL4AuH1c0lDpGIOo6w5KsRS4bhiAJI3eCacwo8enCCfTreUezh/xK190Lube6S8c1q6+ILtRwXeNd2ilYgKWfbeGfd1jyGi9B4BzOj7GUQnD6z8hFGIg/qI+xeEHq8Bt/JSFN25AKj/PHEbWrNFYTDZACbdGMaXbQ0RajyIuvIxWHfaxLDefspJDy07XtbZAVyc2Ezr7y8/UzVrptXw1ywYWM/byJXSOLaDc1pqkqN4NdxJMZUACidGl8n1NfVl4YnEoRV/8P5s0a65fwrn8tucvLh74Iz+378qQGx7h/hEjOH3CCTqN1+ToQL0fqTtHIKx4P8l5m+hw7TbOTF9DmAxiYpdHsIrxJcKbJUaWyvcHrmak16W+ILo3AX6TJmnklaxn3tabiYsoYEdZHF/8eTRpnyfy3HNXEJfYMtDiNTt0oN6E1J0jIDaFsghhFjsAZ6RfrRWKkTRWKt8VgYxL1U2LFherJrqKsXjryjNpkkb7Fj24pvcnJEZOon1UMecNWsLKE6u54+LnAi2apgG0UvEjOjvFzxhZKt9f1H7AK7vrNnWVpR8C/IFCxMKYDteQEHUSsWEVtO+4h7+qysjPKwy0aJp6CLhSEZFRIrJeRDaJyEwXx6eJyB4RWeF8Ta91bKqIbHS+pvpXcvfxRXbKx8tzGTxrERkzP2PwrEWBXcfBbNlUDSmOYJgs2VRrKhTK7DTCgNYTsCsLF/b5nX53rmPkk0/z8qyP8aX73lT3VhARUKUiIlbgGeAMoDcwSURcRanfU0r1d75edp7bGrgLOA4YBNwlIgl+Et0jXGWteIOpFggyYzZVY4rDpG6fgzTVmjKjK88bXAxOOsRmMjHjA0qrMshM2sbkyYt5v9v3nHHeg+Ru3mm4CKa6t4KMQFsqg4BNSqktSqlKYA5wdhPPHQl8rZTaq5QqBL4GRvlITkOoO0egbatIr/prqI6T3zGrC8bsiqMhmmpNmdWV5wkNDE5aRiRzdZ+36RF3Cy3DKji7z59sP83O7Ze9aLgYprq3goxApxSnArXTc3JwWB51OU9ETgE2ADcopbbXc67LnEIRmQHMAOjQoYMBYntO7eJzORvzGH/b3x735fHMYl9k+3jjgjFp9pEpaMpiX6GUYt6EVO/j25xDcdVadpR8RreMHWxISmHDiiy6988wTAw9a99zAm2pNIUFQCelVD8c1sgb7naglHpRKZWplMpMTjZ2vYZA4tHMYl+5qTx1wZjRbRaMBLNFVpsmDk4yEydSbovmnC4rGHzzSs5/+z3eeexTw8TQs/Y9J9BKJRdIr7Wd5tx3EKVUgVKqwrn5MjCwqeeGOh7NLPaVm8pTF4xZ3WaawNDEwUliVBcu7b6ASnt7+sbnovpWMPeD3w0TQ8/a95xAK5UlQDcRyRCRCGAiML92AxFpX2tzLFDjL/oSOF1EEpwB+tOd+5oNHpX19lWmkKfZVM0gc0njBm4MTiKsMfRrfRYWgXFHL2PnROH6S5+hsrzyiLYN4iIxQJfM95yAxlSUUtUicjUOZWAFXlVKrRGRe4GlSqn5wLUiMhaoBvYC05zn7hWR+3AoJoB7lVJ7/f4hAozbCwT5ckGupvj//SmPJvhwMz6UmXQRm4o30SV2EZNH/MhzMpx/jnuUlxfe0bTrNVDKZ9yACVqJeECgA/UopT4HPq+z785a728Hbq/n3FeBV30qYKhhtlpPZpNHE3jcGJxYJZyLut7P5zl3Q9lXJLUvZlNVC0r3lxLTMqbxDsxYAy7ICbT7S+NvzDbpz2zyaIKSoxLGYFMW/tH3Nzr+J5tTrn2Cnz5d1viJ2v1qOAG3VDQBwBM3lS8xmzyNoVOgTUd6i0zGpL/Oh1uvZEjKelafms7DD3zESWMGNnyidr8ajrZUNBp30CnQpqVNdFcGJF6IVRTHdd1EzuBoZj/TSO5OKE0cNQlaqZiIrQf+DLQImsbQKdCmpn/ieZRUd+D45CwumPojD5Qs4/bJT9d/gna/Go52fwWQdp2SaVtUxZactpS03cyqgidYV7SUSZ0fQkQCLZ7GFdoHb2oira24suds3t96A+2i/qBlx30s+bgMpVT991SwuV9NjrZUAkhYeBgffvUfRvzSmVc/G8aWkiSq7D9RWLk50KJp6iPUijeGICJC/8SzALig7xLCbt7PqPP/y/aNeQGWzDWhVg1ZK5UAYw2zcvVtZxP+k4X1hW0BsKuqAEtlcgJZYl/74IOCbq2G0zPuFlqEVTK+z1Lyx5Zz9dRnAy3WEYRiNWStVEzIX4XfBVoE8xLoQLn2wQcNx7U5h6NbX0K4xU6X1F3s7BDFhhVZgRbrMEKxGrJWKiYgOT2RLhXC5s3tKayMZtuBt3l1w3TsqtaPzWwLYAUKMwTKQ6V4YzOgV9woyqpjGN3hL4bc/Cfj583hhXvfD7RYBwnFashaqZgAi8XCW5/fwaV5R/HGe8NZU9weq6zB00JBAAAgAElEQVSloGKdo0GgR+dmQgfKNW7QMqI9l3SfT7W9E33i8og7poi5X6wItFgHCcVqyI0qFRGxisjlInKfiAyuc+zfvhPNXPg6mCYiTLhsBFF/2thSnAiAqlmj3Ayj86bia4tKB8o1bhJhjeHkdlMAGN5lLQcutHLdJR4UnvQBoVgNuSmWygvAEKAAeEpEHqt17FyfSGUy/BVMEwFH0qPj3193f+BQLMEyOveHRaUD5RoP6NpqBBGW0+gYU8C00xbz44m7mTzqwUCLFZLVkJsyT2WQc4EsRORp4FkR+RCYRM3TL8RpKJhm5H9+Qtt4MlvE8sOaFPLabAO+4qX1a7g0IQ1rYRCUkvBHcb5QWuVQ4zcsEsakLvfwy672bNz3Ju1SCsmJSmp64Ukf4nalcZPTFEslouaNUqpaKTUDWAEsAmJ9JZiZ8CSY5om7TER46r0buCf2ZOa8OoRlBR2ItOaye/jFwTE695dFpQPlGg/pHjcMm7JwTpfldJ+ZxSk3PcG37/8WaLFCiqYolaUiMqr2DqXUvcBrQCdfCGU23A2meesuGzlxMK1WlLEuvx0AH1m+ZPuY282fxqrjHRqTkxTVnTHpr1Nlb8HQ1PXEjCzkgccXeNVnqE1e9JZGlYpS6h9KqYUu9r+slAqv2RaR04wWziy4G0zzNvc8KiaSB24+m+IPkvkyuzdWSznfRM9l31ULzT061/EOjS8wOPmjTXRXTk+9CYD+qdspGhzTeOHJegjFyYveYmRK8UMG9mUq3A2mGZF7fvLogXz9v2vJeqITX27vg0UU64u/8UR8/6EnBmqMxkfJHx1jT6Lc1omBidlMmvI9s6r/4JrzH2v8xDqE4uRFbzGyoKRHQXuna+1JHMsJv6yUmlXn+I3AdBzLCe8BLlFKbXMeswGrnU2zlVJjPZS9UdwJpqXER5PrQoG4m3seG9+C1AM2tuUkUZIWwZrCF1lfvIYLu8zCItbGOwgEujifxkh8lPwRbolmRo93+Hz7vcCXpHTdw/L51djtdiyWpo+1Q3HyorcYaakod08QESvwDHAG0BuYJCK96zRbDmQ6M9DeBx6udaxMKdXf+fKZQnEXo3LPLRYLc776F6cv7cCrn5/KpgPJ2NTP7CpbbqS4Go158WHyh4hwbPJ5AIzstIaWVxVzxoQHyd6wo8l9hOLkRW8J9Iz6QcAmpdQWpVQlMAc4u3YDpdR3SqlS5+ZvgOmjvkbmnoeFh3HPExcTvcDG0rwMAD7Nfozy6mKDpdZoTIiPkz/aRPelZ9ytxFirOL/vEkonFnPJxc9is9kaP5nQnLzoLUYqla0enJMK1J6AkePcVx+XAl/U2o4SkaUi8puIjKvvJBGZ4Wy3dM+ePR6I6T7jBqTy88xhZM0azc8zh3mVhx4WHsaU0weQ91MyKwtTCbNs5c1N4yip2m2gxA2g645pAoUfkj+OazOOESn/wSqKjDZ7KOrSgg1/Nq3wZChOXvQWUappXiunq2o0jjTig7EYpZT70a1DfY4HRimlpju3JwPHKaWudtH2H8DVwBClVIVzX6pSKldEOuOYNzNcKdXgYiSZmZlq6dKlnoocULZvzOO8f79GlyuyGJW2huOSb6Nn/NmNn1iDJ2ur1wRKa/u1w6N1AF7jPzz53bpJua2YNzaOJ8pawrp9bfl8fibTKjK4YdaFhl4nmBGRZUqpzMbauWOpLACmAYlAy1ovb8gF0mttpzn3HYaIjAD+BYytUSgASqlc598twGJggJfymJr0bu1pvb2UrbuSqVYWftr1KN/teIsmDQw8zaIJprpjmtDED5Ndo6xxXNztE+yqJz1b7SL9hJ18sHQD5aUVjZ+sOQx3lEqaUupcpdRdSql7al5eXn8J0E1EMkQkApgIzK/dQEQG4Kg/NlYptbvW/gQRiXS+TwIGA2u9lMf0vDn3epLnteTdP4+nuCqK7JLn2FG6rPETPVUOwVJ3TKPxkghrDKPTbwBgULssqsYpbr3qRSrKtGJxB3eUyhcicrqRF1dKVeNwaX0J/A3MVUqtEZF7RaQmm+sRHOVg5onIChGpUTq9cMz2Xwl8B8xSSoW8UmmTlsT8D26n6rkYFm7pC8Bvu+ccqmhcH54qBz1LXtOMSI7qS7R1JOkxhVwy4jv+OC2P8aMewFbdtMC9xj2l8hvwkYiUicg+EdkvIvu8FUAp9blSqrtSqotS6gHnvjuVUvOd70copdrWTR1WSv2ilDpKKXW08+8r3soSTJzYqS271ieRWxbHgepfeH7deCptB+o/wVPlYMZZ8jpxQOMjRIQJne9iYNINRFmr6dAun92JUezdWRho0YIGd5TKY8AJQIxSqpVSqqVSqpWP5NI0wqw3ruLBlCHMe+kkft3dmZiwnWzet6j+EzxVDmabJa8XLNO4wuCBRpeWp2BTFk5L/Zs+123gzPtf4It3fzZI2NDGHaWyHfhLNTVdTONzRl90EslLy1iVm4ZdweKdT7G9ZLXrxt4oBzNVBdaJA5q6+GCg0SK8jaPwpK0lQ1I20PbsnTzw2teUFJcYJ3eI4o5S2QIsFpHbReTGmpevBNM0jXvuOR/bZy35Krs3Fqnkm9wr2FnagGIxi3LwFJ04oKmLjwYabaK7Mj7DUTUqOqICW5SV8tLArxZpdtxRKlnAtzjWVzEqpVjjJSecfjQ/vnALuQ+mMW/DQCyiWFvkWcXVoEAnDmjq4tOBhqOkYVRYNbaWYezb20DcUgO4UVDSgPRhjY+IiIogwxrOyh3x7O8WyfaSD3ljYx7/6PIgVktE4x0EE8PvdD0ZU5fXb77EpTldXy72e0mriHQq7RGc2n498deUcu5rNm5I6c+UG8d43Xeo0mRLRUSSReQREflcRBbVvHwpnKbpvPLZbZy1JpU3PhvC+n1tgV/5u/jzQItlPGZLHNAEHh9mKEZZ45nUeR4VtlQyk7LJGJnLC9/8SdEeXXuvPtxxf70DrAMygHtw1Ppa4gOZNB4QFh7Gff83ndbvV/PdJkcxu192vRWahSdDITakMQ4fDzRiw5O5oPMDAPROzKPsJAvPP/QJdnsjc8NMQCBWpXSn9tcypdRAEVnlLEOPiCxRSh3rUwkNJphrfzWFd574jIe3r+LUCcs5OiGHSnskEzJeIyGyU6BF02iCFqUUb226DsVS9lVH8vbvg0l/J5L3Ft6B1WrOtY1qVqWsvYhYdLjV44KXvqj9VeX8mycio53lU1q7LZnGp1x0/Wg+u3Iavz90FB9nDSDKWsGaohB0g2k0fkREmNLtKXrG/5NWYRV0y9jB9vgw8jbv8tk1vbUyArUqpTtK5X4RiQNuAm4GXgZu8IlUGq9I69qetD3VZOckU2G3sr7oXb7b8WbTCk9qNJp66ZdwBnYlnNhmCx0u3cYF97zKip/+Nvw6NVZGblEZCsgtKuP2D1e7pVgCtSplk5SKs+x9N6VUsVLqL6XUqUqpgTWlVDTm45nZ15D+eSTvLT+OoqposkueZ23xZ4EWS6MJaqLDEhnc9lFQYYzpsoqkKXlcff88wwP3RlgZgVqVsklKRSllAyb5VBKNodQUnrQ+Fc2cVccBsKJgfuOFJzUaTYN0jzuBiV2eByA+upSqlmEU7fa6DOJhGGFlBGpVSnfcXz+LyNMicrKIHFPz8plkzQB/ZGYMGdCJsi0tyC5NoNr+F/+39hx2lW00/DoaTXPCKuEAiCjsYRbDqxgbYWUEalVKd7K/vnOxWymlhhkrkm8xS/aX0ZkZDfHF7J+5c/Ei+kzazOD2m7GKnZZhozkv41Ys0uT5r0fihxX5NBozUmHbx9ubRmMHfs7rwprZXbj/1OGMnHiiIf378/nQVAzP/nLGUeq+gkqhmAl/ZmacMWkw3z96E5Ev9+f1xaeQU5pAqe1TluW/63mn3hbx0+XrNUFMpLUVY9JfocLWilNTN9Bpwnbueedbdm/PN6T/QFkZRuDWMFVERgN9gKiafUopXR7WA/ydmRHTMoYX3r6esYP/w/uRx3LjiV+yqvBTjkm8AKsl0v0OGyri5+669zUKCbSlE0qEuCXbNqY7U7q+wEfbJtAysgJbizD25e+nTXqSR/19vDyXR75cz46iMlLio7llZA9+nhl843Z3yrQ8D1wAXIOjytr5QEcfyRXyGJaZ4eaIf9zoY2BdJH/va0eY5PDC+rMoqtjm3jXBuyJ+unx96NNM1r0RcTxCrRY7tggLZSXlHvVjRAqxWXAnUH+iUmoKUOgsLnkC0N1bAURklIisF5FNIjLTxfFIEXnPefx3EelU69jtzv3rRWSkt7L4E0MyMzy4cS+54xzeGnsOi58YyKdb+xFtPcAfnrjBvKkWrMvXhz7NZOAQZU2g0h7FCUlbOP6KFUz95APeeHSB2/0EaqKiL3BHqdT8QkpFJAXHDPv23lzcOf/lGeAMoDcwSUR612l2KQ5F1hV4HHjIeW5vYCIOd9wo4Flnf0GBIT5TD2/co0/sQbdcG39vSaPCHkb2gc9ZUfCVex/AmyJ+unx96NNMBg7hlmgmZsyhrDqNY5O30XtMFs/+uoqcDTvc6idQExV9gTtK5VMRiQceBpbhKCg528vrDwI2KaW2KKUqgTnA2XXanA284Xz/PjBcRMS5f45SqkIplQVscvYXNIwbkMrPM4eRNWs0P88c5n4Qzosb9/5nL6HNV7BgQ3/KbOGs3Hs3K/d+1PRre1PEz1OFpIP7wUMzGji0jGjDtO5PAxAdVokt0sr+QvdWiAzUREVf4I5SeRS4BJgM/IpDuTzg5fVTcSxTXEOOc5/LNkqpaqAYSGziuQCIyAwRWSoiS/fs2eOlyCbCixu3U680vpl3B/JoAq/+cRLVysKaQjdrhHlaLdgThdRMfPQhgw/L0bsiENV4XREZVo09VijYVeTWeYGaqOgL3FEqb+BwNT0F/B8Od9WbvhDKaJRSLyqlMpVSmcnJyYEWxzi8vHEtFgvHdk+hMjeG3RUtqbKv4dm1/6Cs2r0bwiPcVUjNxEcfMvhx3RszBLmjLHFU2FuRmbCN02b8wXUrF/LQLU1/PAZzCnFd3Ekp7quUqh3v+E5E1np5/VwgvdZ2mnOfqzY5IhIGxAEFTTw3tKm5Qb1I27zzpRmkPTKfZ96p5sSz1zAwcQs/7HqOkam3+0hoD2kmPvqQot8Ev6QQNxTk9tdD2WqJYFrX93lz0830jlvF7qGxfPBMGOetyKJr/4wm9TFuQGpQKpG6uGOp/Ckix9dsiMhxgLdT05cA3UQkQ0QicATe6xapnA9Mdb4fDyxSjjIA84GJzuywDKAb8IeX8gQfXi5YJSJceuvZ9P7dzuKVfaiwh7F1/7cUVrhYntUHNNlt0Yx89Br3MEuQO8Iayz+6PAhAZJgNe4RQtt+zFONgxh2lMhD4RUS2ishWHHGVY0VktYis8uTizhjJ1cCXwN/AXKXUGhG5V0TGOpu9AiSKyCbgRmCm89w1wFxgLbAQuMpZ+DLk8YX/+KpbziL6N+HXnV2wSjkfbJ3E/OwnfVqA0i23hZ999JrgIZSC3KGAO7W/GpzoqJTyYAad/wlk7S9XM2bdNXd9WRNo394DXHXl82wdVcCZA1eQEl3M/qpEJnd9jrgI4y2CwbMWketiNJkaH+16JnGIz9DWeIaZ6mSVVxfyXtZofs3vzJIXjiImIoWc1q09vt/NRFNrfzU5phIsSsOs1P3h14zKAbd+aL70H7dqHctb793MN+//xsxnYuk9aRNDUjbwXd7TjOs4y6u+XeG228JPPnpNcFHzu/d2wGYEYZZoqu3hDEjIpmRKJGverCRmSwm5ndM9ut+DEXfcXxovMGrGrD/8xyPGH0/mdgvLVnWj1BbJnvJfyNr/p2H916DdFhqj8HrOl0GEWaIYmfYsBypiGJH+N72mbSXWVkB4QVHQzpB3F61U/IRRysBfD+Lr75tA/E92Fmd3x6aExXlXs7zgA0OvEUq5+RpNDWkt+vDKB9MBiIsqo6pVGGH7S4HgnCHvLlqp+AmjlIG/HsQ9B3bhu7duo/rBdrzyxylU2MNZU2js6tGhlJuv0dSmTasYl/ubgxXuxQpNGne4ZWQPl8FEd5WBP/3HEZHhHNcjhXe2FbJ3QAzR1o28tO5qpnabRYQ11pBrhEpuvkZTm+uGd6XuAsPNxQrXSsVPGKkMfPkgrpuhdtP0UUyc/R0ffno8w05bRe9Wf/J5zgOM6/igT66v0XiFSTIER/dLYfaWQ9tJsZHc1EyscK1U/IjZR+WuMtT+NX8tD04ewc7/vM2n0Zmknv4NlfYllFYXERMWH2CJNZpamHjxtycn9ucYE9/7RqJjKpqDNJShdsH0U4n6C9YUphBuKeOtTePYsv/XAEmq0bhA14czBVqpaA7SUIbaKeMG8dm/L2Hjf7sxe80gwM4vu1/yr4DBiC7X7z90fThToJWK5iCNZai179SWIUmJ5PzdjuLqaGz2dXyd+wpNrcrQ7NDl+v2Lrg9nCrRS0RykKenKN/xvMr2XKT5bcTS7y1uyo/QV/m/teeSXZflbXN9gpGWh3TGH42urzdv6cNqqNAStVDQHacq8kbikVrz/wUyuLR/JnBdO5se8rsSE7eaT7MmsKHBjHosZb2CjLQvtjjmEP6w2b9Zw0ValYejsL81hNDVD7bzpwxlZfDyTzn+M16akcOHA31i+9236J45t9FzTZuk0ZFl4IldcmvMh5WJ/c8Po77Y+PK0P5yf5jCgqa3a0pWJCzLI0amPExrVgRN+OFG1szd6qFljIYfbmu7GpqoZPdMct5E+LxmjLQpfrP4TZrTY/yGeGFSr9gVYqJiPYfnhXPHABQ7dG8dmPA8gqSaTS/hUfbr2t4ZOaegP72yVhdKDXj0vqukUgXI9mD6L7QT6jisqaHa1UTEaw/fAioyN55o1rGf5LW9774iR2V8RSXLmCKlsDK9419Qb2d6DbF5aFlytzGk6gYgdmt9r8IJ9ZVqj0NQFTKiLSWkS+FpGNzr8JLtr0F5FfRWSNiKwSkQtqHXtdRLJEZIXz1d+/n8A3BOsP77RzBxGVZSdrfxLhlnJeXD+GNYWLXTdu6g3sb5eJWS0LIwlURpo/vltvLDA/yNdclnoIZKB+JvCtUmqWiMx0btf1m5QCU5RSG0UkBVgmIl8qpYqcx29RSr3vR5l9Tkp8tMvVEM3+wxt63nG8k57Ilf+zkTclgRGd17I0/w5+2nU0U7s+RFRYq0ONa27Uxmo0BSLQHeoLgQUytuHL79aI5A8f/98bVVTW7ARSqZwNDHW+fwNYTB2lopTaUOv9DhHZDSQDRQQJ7mZ7BNsP74gClLdMZf3cxby2vC1DR66iT9xK5m29nsldXz38xKbcwMPvPPxBAeZymQQjoZqR5q/sMi8w0wqVviSQSqWtUirP+X4n0LahxiIyCIgANtfa/YCI3Al8C8xUSlX4RFIP8WQJYVP/8OpUgF3S5RpuX9Lx8AKUn6zhwUnDmPDSQl5bEE3r8d/RLmojJdV7aRHW2r3rNdWicUPmZr+ufagqapNll+WWrnW53+xFZY1AfFliQ0S+Adq5OPQv4A2lVHyttoVKqSPiKs5j7XFYMlOVUr/V2rcTh6J5EdislHLpGBaRGcAMgA4dOgzctm2bx5/JHQbPWuTSlZUaH83PM4f5RQbDqOteAMqI5LbKS5lvP+mwpqnx0Tx/cgqXPPMhvWZsZGjKBqrsVnrEzWBIu4sQkYDJTHh06MVJ3CUUFe3jfeuxwNIdCRJ+ZEH2w+SXf0Kl3cq8tZmoZ1sx/4NbaBHXwq9yGI2ILFNKZTbaLlB1m0RkPTBUKZVXozSUUkf4eESkFQ6F8t/64iciMhS4WSk1prHrZmZmqqVLl3ole1PJmPkZrr5dAbJmjfaLDIZRz02bY0/ipMqnDttX8/l2Zedz6Y0vUXF+PqP6riIpooR9lWlM7vY48RF+GK2Z6EGj8TEmGUBU2PYxZ8socsvi+OTHQQz8OZ4nXr2KiKgIv8ngK5qqVAKZUjwfmOp8PxX4pG4DEYkAPgLerKtQnIoIcQx7xwGme0oEQ7ZHkyda1uNGSJGCI/c5P1/bDkl8+v7tXLLnNN59ZQi/7OpMTNgOPtx6vWHyN4jJXCIaH2KSzD2lHK7g7JJEbL9Hc+WVo0JCobhDIJXKLOA0EdkIjHBuIyKZIvKys80E4BRgmovU4XdEZDWwGkgC7vev+I3jr/XkPcWtiZb1BHLzSDxs29Xnm3T1KC6P6cpPnx3N9vIErJLL7rLN+ByzT7jTGIsJ5gT9XfyT369pNgKmVJRSBUqp4UqpbkqpEUqpvc79S5VS053v31ZKhSul+td6rXAeG6aUOkop1Vcp9Q+l1IFAfZb6aEqBxkDi1kTLeuaW7Bh4a5M+3xlTTiZ+bTmrdqZRabeyIHsKH2Q9hF1VG/iJmiZz0AelNaZkzpbbWLX3QQ5UR7ImK5X2OyrocWyXQIvldwIWUwkU/oypmB23Yz5eBnhL9pVy3RUvsHZoPmcct4IOMYXsq4zn3E4Pk9qir8efo0FCMSitMR2l1fnMyxpLVkki878cxFnZ7bnzqYuxWq2NnxwkNDWmoqsUN2Pcnmjp5eSwFq1iePndG/j5iz+58Ylouly0lSFpG/gi51am9/jc434bJNQnM2oCTkl1Pm9uuI6YcNhUnEzYEivTHzgjpBSKO+jaX82YQMV8Bp9xDDf2G8DqL3qQXdqacEsRG4t/8+k1NRpfUG0v593NFxAZtpU/C9JZszCD6cf0JL2HOVzcgUArlWZMIGM+p006kXZry1i6LYOS6gh+3nUjr6y/nkpbic+vrdEYxa6ytURYyvg9P4Ofnh/A66POZcYd5wZarICiYyqagGGrtnHfTW/yWUo2w4avpHfcTkqqozgh+VYGJI3yqu/msBiSJnAoZWdh7gvklryLiOK9dcdS+WQrPvvwVr7eUhSSvz3TT34MFFqpmI+Nq7ZxxX1vE/OPXZzW7S/iwsspqerFtO6PEhPmsshCg9QtjwMOt56ZMu80wcvusk3M2XITcRF72Fneii+WH03bBa15/pkr+D73QMj+9oJh8qNGA0C3fh35+r3bOe3Pk3hjzlBW7E0jOmwdb20ax0875+DuwCfY1qTRBAd2Vc37WQ+yIHsq0WF7+X5Hd95/7iRutozmvXm3kdAmTv/20NlfGpNgsVi49p4JTNi6m0tvepk1F6Qxqs9qNu9/iuUFHzOl2xPERbgqI3ckDa5Jo1OMNR6QfWAln2ybSauIYnaUJ7Dwt/70+j6J71+4nJiWMQfbBet6SEailYrGbXwZr2jXqQ2fvn87bz/5OU/8kcCgc9aRmbyVuVnn0zPuck5u949G+6gvVXpq7B+w4AXv1txwhRkUlRlkCEGqbeW8u+UebOp7wq1hLNzWm6x3OvHYhDM58d1jjmgfrOshGYmOqQQxgQhG+zNeUbi7mOkzniV/fCFnD1hOQngpU7t9j0UaHgvVJ+Oy2OuJKcs78gRvCkyaoZChGWSoh2BOmPi78AcW5d1LbHgpG/cn8/UPR3Py3+3571PTiYgMd3lOKMfzdEwlxHGrbpeB+NNnnNAmjn9dPZr9ixLZvD8Ziyg+3vY/7MrW4Hn1pUrHlO10fYI3BSYDtTyv2WRwQaB+o95SYTvAi+uu4o/8mSDVfLKhPz88ciyvnDCZR1+4sl6FAuYvzeQPtPsrSGno4e7LH7C/fcZHn9qH417+liXLupFychFpfMLTa3/gnE4Pkd5AaReXiyEt9sGqh2aohGwGGVwQqN+oNyzds4Al+Y8RZa3gr6L2LPq6H+fuzWDm2xc1eYZ8c1iIqyG0UglSAhUQ9LfP2Gq18tycG1j8yVJuezKcbhdmcXLqRr7JvRyrDKZP6+OJkEh6x4/Aaol02UeNCyZz31nMiniFaGotEOptgUkzLM9rBhlcEAxB65KqAtbv+wG7svH77vnEhm+iUkWzYM2xVL2TzPv3TiWjd3qgxQwqtPsrSAnUWi2BKu0y9OxMfnzmVhLfPYZXvx5KVkkiip/4a++j/FnwAM+vG8264h+POK+2C+YT+0ncVnkpuSoJZdSaG2aohGwGGVxg5vWElFJ8k/sa72w+h9V7H2FN4WNEh21mWUEH3nxnKGP+OoWv5v1LKxQP0IH6ICWQAcFAB1+X//g3Vz//AZZRBwiPqCYpYT9D0tcTG1ZJpe0YpnSbRaQ1FvDTks5myLwygwx1MGvQuqBsG+9suZG4iDx2V8SyeEtPysoiKNsXSfLCWF5+8jLapCUFTD6zomfU10OoKBUI/MM9kNiqbSya9xsH9pWybOU2FqbnMuy0VfRulUeZLZJjk28iM2lMaC3pHISY4TdqVzYWZD/F9pKfAEW0dTcK+H13Bsvm9WR66260b59AaqdkBo3s31h3zRatVOohlJSK5hBb1mQz4+43ibxoDyO7ryYuvJwDVd35cOG5bN59ZIDVUEtFYzrySteytugHbKqa9UWf0yqiiKKqaKqVhf1VUXz7Z18yvkrimecup2VCbKDFDQpMr1REpDXwHtAJ2ApMUEoVumhnw7FkMEC2Umqsc38GMAdIBJYBk5VSlY1dVyuV0EUpxbP3fcBrlWsZPGYNR7fOocpuZdHyoazb1BOb3cKBsliiw8MC7oLRGItdVVNWXYBNVfHxtqexqR+wiONYuS2MH3K789f8zljKhJh9du67YATDzjkusEIHGcGgVB4G9iqlZonITCBBKXWbi3YHlFJHDCVEZC7woVJqjog8D6xUSj3X2HW1Ugl9dm3PZ/qNL1E+Pp9RfVeRFHGonH7OvmS6yr+44JhBAZRQYySrCxfxY95/aRFeenDfxv3JfL+mF9WVYZTvjmTIptY8/MxlRERFBFDS4CYYlMp6YKhSKk9E2gOLlVJHpBC5UioiIsAeoJ1SqlpETgDuVkqNbOy6Wqk0H2Y/vZD/bV5K1zOziY6oJDaynKMTHHM32kSNZ0yHa7CIiVbnM2Gw3YyUVTQHT4cAABEXSURBVBfx9qZ/UWHPQrARG76f/dWRrMhPx2a3sLuoFftnt+GOs4bQKj6GDt1TSOvWPtBiBz3BoFSKlFLxzvcCFNZs12lXDawAqoFZSqmPRSQJ+E0p1dXZJh34QinV6ELnWqk0L4oL9vO/O+ZQUFzKjvIy8scWc0bmClKji9lXmUD7FoOwYOX45HNJadE7cIKauNSKX2hAoSqlWL73U7bsX061vYKiip+JtFaysyIOuxL2lMXy01d96bcqlmirlT69Urj83+f6dTlfMyQk+BpTKBUR+QZwVVr2X8AbtZWIiBQqpY5YPENEUpVSuSLSGVgEDAeKcUOpiMgMYAZAhw4dBm7bts3LT6YJVr794HduX/gVPSZlcVLKJqKs1QDYlBBhOZUz0iYjYiEuohNWqb8ch+E83reeCYxe1CUzA02xvmop1PJIK6Ux4RAWCafcSnmXwXyQ9RCtIrIPNi+ojOHLv4+icEESUg3tSiw8/cg00uuzRnxsAZo1ddpoTKFUGrxwE91fdc55HfgU+ADt/tJ4SFlJOTf/8yV+7LwPibUTHVfJ6QNX0iU2/2CbA1UtGJ56Jz3jTvaPUHfHQ33Jz3cX+UcGo2mq9fV4X+z7trOgfzf29Iwi3GI/rJtqZWHpno78urQHqlqwZFu5rkM/pt44GoeTwwAZvMAvc6FMQFOVSiDLtMwHpgKznH8/qdtARBKAUqVUhdPlNRh4WCmlROQ7YDyODDCX52s0rohuEcUzb1xDfm4B+woOsCM7n9sejWTJeXtoFVtGVEQVg9pm8fvu21i8IxIQrNKLyV0fJCqslW+EMmmpFa+op9DltmUPMt/6LuGWfY59IxOxWBKItNrYXdaS1bvSsdfSrztyEkn5Ip65M8cTFh5Gu4w2xLRs4qz8hoptGqRUgqEcjT8JpFKZBcwVkUuBbcAEABHJBK5QSk0HegEviIgdR0mZWUqptc7zbwPmiMj9wHLgFX9/AE1wk5SaSFJqIp37deSHUf15/dEFbN+WT/7+El7r04kTT/2buMgywi020qOX8/rGsUSGHYUFK33ih5GZPNY4YYbf6XpEHeBSK4fRRDdSpa2ET3OeZP+AeODwMGlZRBiRSZWE2/eSW3a4tztrbxIb3u3AyE3LsYoNwmOg62lccXo/Tp3nYbaewcU2XcVO9Boqh6MnP2o0LtiyJpub7niHQmxURkHcBfmM6LGGlmHlWFCEW+zsr+zEaWnTCXMRe2kb3YPY8DbuXdTM2V913Eh5iTGUtoiFzEuh0+CDzXaWbmNl4avEWMupVK4C5UJ2SQJf/3A0Ud9HIHZA2cBWyYCSTTx44ldERdiNc1EZGKuqL3Zy3sBUPliWq2MqNe20UtFoXHNwVFpYStq6jZSklVHdFiyRNgafvJaBSdlYxfX9U2W3eJW2HOhsorrX/1r+SUxZHvujw3ljaD9ati6v99ziqii+2tiX7JWul39OWFXJ0zdN4qgTaoVQfaVQDYypNBQ7uWVkD539VdNOKxWN5khcjkpR/OvUDE7uGMdN/3mbvFH7aJ28/4hzrRY7gzpsITW6GLsCuzqyGHhpdRxjOtxHRssjl6T1ZzaRXVXz4dZH2FvxBVaxN9o+zGKnWgnL8zuwLf/Ioot2JexcmsQ0ay8mzRgOaxfAT4/D/jxo2R45+QZaD5tyMMDuF+VpkMJq7nXktFKpB61UNE2hKRk9Py5YytplWUe0KS2vYnbBZrpesI32rY7M3LIIdGm5h0hLFRW2WBQphx3PKy6j2n7kfRlmEdrHOf30lSVQXgj2arCEQVQCRLRw+3Pa1Q5iww+QWxZPQXlM4+2xsGJDJ1q/XsrpllUQ2RIGTjt43BpmYczkU2jXMbnRvoItFbe5ZHnVRzBkf2k0pqUpGT0nn5XJyWe5vseuLC3nP9e+ztrde484ZkPx/dBKTjhxHR1b7iXccvi8qeS4+uWqrnkT4XwhgA3Id77cw6asLNzWm83vZpCc77AeCkuOLKEXQRUxVBCuqrmu9TeMO2lzLTfS+W5fF4JvZchbRvZwqQR9vZZQsKGVikbjAm8zeqJionjk5SvqPb52ySbueGAeP3RU2AN4F0qFMKIynjeem35w7fX6RuTTYv/g7hYfON1I6V7HPYItFbdG0YV67MRbtFLRaFzg61Fp72O78vHHt1NVWXXEsfkrdvDvj/864tr3j+vL2P4p/9/e3cfKUZVxHP/+2nJLCZG2grUW6ItBkFDTktKgGBFseBGhjTRYlFC0hBdREo1Am2qCGEI1MSSKiEiAKqYUqoUqqdjSohEBKaH0BdIXaAhcS1vAYhSoFR7/mHPpdLt77957Z2d3298n2dzZmTN7nzm7mWfmzMw58IMPUfNBye9t71UcAwYMYOCgvW8kqLXtE865DCZ+v1ef3512vBX3QB9/vh5OKtbSmnUXVFlHpQd17Hs78vmTRzPwoEG1//fwUbUflKzyeb1V1ra7OWn/5Av11rLa7UJuafajziebfeu01c8X6q3ttduF3NJ0JY5WfVCyF9yctP9xUrGW1W4Xckv1iQvaMonY/m/fp7LMWkStC7atfCHX7EDnpGIt65ozj2XIQXvfmeQLuWatzc1f1rL8XIBZ+3FSsZbmC7lm7cXNX2ZmVhgnFTMzK4yTipmZFaZpSUXScEnLJG1Kf4dVKXOapNW51zuSpqVld0vakls2ofytMLMyPfBMJ6fMW8HY2Q9xyrwVPPBMZ7NDsgrNvFA/G3gkIuZJmp3eX5cvEBErgQmQJSFgM/CnXJFrImJRSfGa9Zu7Jem7ym57One+zZzfrQVwHbaQZjZ/TQXmp+n5wLQeyk8HlkbEWw2NyqxBunaKnTvfJtizU/TRdn2667bHWkczk8qIiNiapl8FRvRQfgawoGLejZLWSLpZ0uDCIzQrUCvsFNu5+cjd9rSHhiYVScslravympovF1lXyTW7S5Y0EhgPPJybPQc4DjgJGE5F01nF+pdJWiVp1Y4dO/qzSWZ91uydYrufKbnbnvbQ0KQSEVMi4oQqrweBbSlZdCWN7kYXugBYHBHvj2gUEVsjswu4C5jcTRy3R8SkiJh0xBE9j51t1gjN3im2wplSf7jbnvbQzOavJcDMND0TeLCbshdS0fSVS0giux6zrgExmhWm2TvFZp8p9de0iaO46YvjGTV0CAJGDR3isXVaUDPv/poH3CdpFvAS2dkIkiYBV0TEpen9GOAo4M8V6/9G0hGAgNVA7QHBzVpAs/sya8fheyu5257W55EfzQ4QHknT+sMjP5rZXpp9pmQHBicVswOIm4+s0dz3l5mZFcZJxczMCuOkYmZmhXFSMTOzwjipmJlZYZxUzMysMAfcw4+SdpA9wV+mw4HXSv6fvdHK8bVybOD4+qOVYwPHV2l0RPTYeeIBl1SaQdKqep5EbZZWjq+VYwPH1x+tHBs4vr5y85eZmRXGScXMzArjpFKO25sdQA9aOb5Wjg0cX3+0cmzg+PrE11TMzKwwPlMxM7PCOKmYmVlhnFQKImm4pGWSNqW/w6qUOU3S6tzrHUnT0rK7JW3JLZtQdnyp3Lu5GJbk5o+V9KSkzZIWSuooMzZJEyQ9Lmm9pDWSvpRb1pC6k3SWpA1pm2dXWT441cXmVDdjcsvmpPkbJJ1ZRDy9jO3bkp5LdfWIpNG5ZVW/45Lju0TSjlwcl+aWzUy/hU2SZlauW0JsN+fi2ihpZ25ZGXV3p6TtkqoOka7MT1L8aySdmFvW0LqrS0T4VcAL+BEwO03PBn7YQ/nhwBvAIen93cD0ZscH/LvG/PuAGWn6NuDKMmMDPgYck6Y/AmwFhjaq7oCBwAvAOKADeBY4vqLM14Hb0vQMYGGaPj6VHwyMTZ8zsOTYTsv9tq7siq2777jk+C4Bbqmy7nDgxfR3WJoeVmZsFeW/CdxZVt2l//EZ4ERgXY3lnweWkg2lfjLwZBl1V+/LZyrFmQrMT9PzgWk9lJ8OLI2Itxoa1R69je99kgScDizqy/pFxBYRGyNiU5r+B7Ad6PHp3n6YDGyOiBcj4r/AvSnOvHzci4DPpbqaCtwbEbsiYguwOX1eabFFxMrcb+sJ4MgC/3+/4+vGmcCyiHgjIv4JLAPOamJsFwILCvz/PYqIv5AdcNYyFfhVZJ4AhkoaSePrri5OKsUZERFb0/SrwIgeys9g3x/rjel09mZJg5sU38GSVkl6oqtpDvggsDMi/pfevwIUOXxgr+pO0mSyo8wXcrOLrrtRwMu599W2+f0yqW7eJKuretZtdGx5s8iObLtU+46LVG9856fvbJGko3q5bqNjIzUZjgVW5GY3uu7qUWsbGl13dfFwwr0gaTnw4SqL5ubfRERIqnmvdjqqGA88nJs9h2yH2kF2//l1wA1NiG90RHRKGgeskLSWbGfZLwXX3a+BmRHxXprd77rbX0m6CJgEnJqbvc93HBEvVP+Ehvk9sCAidkm6nOyM7/SSY+jJDGBRRLybm9cKddfSnFR6ISKm1FomaZukkRGxNe34tnfzURcAiyNid+6zu47Ud0m6C/hOM+KLiM7090VJjwITgd+SnWIPSkfkRwKdZccm6QPAQ8DcdNrf9dn9rrsqOoGjcu+rbXNXmVckDQIOA16vc91Gx4akKWRJ+9SI2NU1v8Z3XOSOscf4IuL13Ns7yK6rda372Yp1Hy0ztpwZwFX5GSXUXT1qbUOj664ubv4qzhKg626LmcCD3ZTdp5027Uy7rl9MA6re+dHI+CQN62o6knQ4cArwXGRXAVeSXQequX6DY+sAFpO1JS+qWNaIunsKOEbZXW8dZDuYyrt98nFPB1akuloCzFB2d9hY4Bjg7wXEVHdskiYCvwDOi4jtuflVv+MCY6s3vpG5t+cBz6fph4EzUpzDgDPY+4y+4bGl+I4ju9j9eG5eGXVXjyXAxekusJOBN9OBVaPrrj5l3xmwv77I2tIfATYBy4Hhaf4k4I5cuTFkRxQDKtZfAawl2yHeAxxadnzAp1IMz6a/s3LrjyPbMW4G7gcGlxzbRcBuYHXuNaGRdUd2l81GsiPRuWneDWQ7aoCDU11sTnUzLrfu3LTeBuDsBvzeeoptObAtV1dLevqOS47vJmB9imMlcFxu3a+lOt0MfLXs2NL764F5FeuVVXcLyO5u3E12XWQWcAVwRVou4Gcp/rXApLLqrp6Xu2kxM7PCuPnLzMwK46RiZmaFcVIxM7PCOKmYmVlhnFTMzKwwTipmZlYYJxWz/YSkb6Tu0CM9nGdWOicVs5KlLl0a4TFgCvBSgz7frEfu+8usD5QNyPVH4GmysS/WAxeT9Tt2LjAE+BtweURE6idqNfBpYIGkjcB3yTrBfB34SkRsk3Q9Wc+444CjgW+RjZlxNllPDOdGrs+4vIh4JsVW+Paa1ctnKmZ9dyxwa0R8HPgX2aBdt0TESRFxAlli+UKufEdETIqIHwN/BU6OiIlkY3pcmyv3UbIee88j63ZmZUSMB94Gzmn0Rpn1h89UzPru5Yh4LE3fA1wNbJF0LXAI2Qh868m6eQdYmFv3SGBh6lixA9iSW7Y0InanYQcGkp0RQdbP05hGbIhZUXymYtZ3lR3nBXAr2dDG44FfknU62eU/uemfkp3VjAcuryi3CyCy8WJ2x54O+t7DB4LW4pxUzPruaEmfTNNfJmvSAnhN0qHsGSqgmsPYM47HzG7KmbUVJxWzvtsAXCXpebKxN35Odnayjmwci6e6Wfd64H5JTwOvFRGMpKslvULWtLZG0h1FfK5Zb7jre7M+SHd//SFdkDezxGcqZmZWGJ+pmLUZSYvJnmXJuy4iyh861qyCk4qZmRXGzV9mZlYYJxUzMyuMk4qZmRXGScXMzArzfxzcy8JnJeEpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the number of training examples.\n",
    "num_examples = x_train.shape[0]\n",
    "\n",
    "# Set up how many calculations we want to do along every axis. \n",
    "samples = 150\n",
    "\n",
    "# Generate test ranges for x and y axis.\n",
    "x_min = np.min(x_train[:, 0])\n",
    "x_max = np.max(x_train[:, 0])\n",
    "\n",
    "y_min = np.min(x_train[:, 1])\n",
    "y_max = np.max(x_train[:, 1])\n",
    "\n",
    "X = np.linspace(x_min, x_max, samples)\n",
    "Y = np.linspace(y_min, y_max, samples)\n",
    "Z = np.zeros((samples, samples))\n",
    "\n",
    "# z axis will contain our predictions. So let's get predictions for every pair of x and y.\n",
    "for x_index, x in enumerate(X):\n",
    "    for y_index, y in enumerate(Y):\n",
    "        data = np.array([[x, y]])\n",
    "        Z[x_index][y_index] = logistic_regression.predict(data)[0][0]\n",
    "\n",
    "# Now, when we have x, y and z axes being setup and calculated we may print decision boundaries.\n",
    "positives = (y_train == 1).flatten()\n",
    "negatives = (y_train == 0).flatten()\n",
    "\n",
    "plt.scatter(x_train[negatives, 0], x_train[negatives, 1], label='0')\n",
    "plt.scatter(x_train[positives, 0], x_train[positives, 1], label='1')\n",
    "\n",
    "plt.contour(X, Y, Z)\n",
    "\n",
    "plt.xlabel('param_1')\n",
    "plt.ylabel('param_2')\n",
    "plt.title('Microchips Tests')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
